A Starter Recipe

This module is designed to get you thinking about how to strategically approach training a deep learning (DL) model. Although finding optimal hyperparameters for a DL model includes a lot of artistry and patience, using a methodical approach can reduce overall model exploratory time.

The following steps provide a good mental model for tuning our model. Realize this approach does not guarantee you’ll find the optimal model; however, it should give you a higher probability of finding a near optimal one.

1. Prepare data
2. Balance batch size with a default learning rate
3. Tune the adaptive learning rate optimizer
4. Add callbacks to control training
5. Explore model capacity
6. Regularize overfitting
7. Repeat steps 1-6
8. Evaluate final model results

We’ll demonstrate with one of the most famous benchmark data sets, MNIST. We’ll continue working with a multi-layer perceptron (MLP); however, realize that these steps also translate to other DL models (i.e. CNNs, RNNs, LSTMs).

Package Requirements

library(keras)       # for modeling
library(tidyverse)   # for wrangling & visualization
library(glue)        # for string literals

MNIST

keras has many built in data sets (or functions to automatically install data sets). Check out the available datasets with dataset_ + tab.

We’re going to use the MNIST data set which is the “hello world” for learning deep learning! ℹ️

# c() %<-% dataset_mnist()
mnist <- dataset_mnist()
str(mnist)

List of 2
 $ train:List of 2
  ..$ x: int [1:60000, 1:28, 1:28] 0 0 0 0 0 0 0 0 0 0 ...
  ..$ y: int [1:60000(1d)] 5 0 4 1 9 2 1 3 1 4 ...
 $ test :List of 2
  ..$ x: int [1:10000, 1:28, 1:28] 0 0 0 0 0 0 0 0 0 0 ...
  ..$ y: int [1:10000(1d)] 7 2 1 0 4 1 4 9 5 9 ...

c(c(train_images, train_labels), c(test_images, test_labels)) %<-% mnist

Our training images (aka features) are stored as 3D arrays

• 60,000 images consisting of a…
• 28x28 matrix with…
• values ranging from 0-255 representing gray scale pixel values.

# 60K images of 28x28 pixels
dim(mnist$train$x)

[1] 60000    28    28

# pixel values are gray scale ranging from 0-255
range(mnist$train$x)

[1]   0 255

# train_images[1,,]

Check out the first digit

digit <- mnist$train$x[1,,]
digit

      [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10] [,11] [,12] [,13] [,14] [,15] [,16] [,17] [,18] [,19] [,20] [,21] [,22] [,23] [,24] [,25] [,26] [,27] [,28]
 [1,]    0    0    0    0    0    0    0    0    0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0
 [2,]    0    0    0    0    0    0    0    0    0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0
 [3,]    0    0    0    0    0    0    0    0    0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0
 [4,]    0    0    0    0    0    0    0    0    0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0
 [5,]    0    0    0    0    0    0    0    0    0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0
 [6,]    0    0    0    0    0    0    0    0    0     0     0     0     3    18    18    18   126   136   175    26   166   255   247   127     0     0     0     0
 [7,]    0    0    0    0    0    0    0    0   30    36    94   154   170   253   253   253   253   253   225   172   253   242   195    64     0     0     0     0
 [8,]    0    0    0    0    0    0    0   49  238   253   253   253   253   253   253   253   253   251    93    82    82    56    39     0     0     0     0     0
 [9,]    0    0    0    0    0    0    0   18  219   253   253   253   253   253   198   182   247   241     0     0     0     0     0     0     0     0     0     0
[10,]    0    0    0    0    0    0    0    0   80   156   107   253   253   205    11     0    43   154     0     0     0     0     0     0     0     0     0     0
[11,]    0    0    0    0    0    0    0    0    0    14     1   154   253    90     0     0     0     0     0     0     0     0     0     0     0     0     0     0
[12,]    0    0    0    0    0    0    0    0    0     0     0   139   253   190     2     0     0     0     0     0     0     0     0     0     0     0     0     0
[13,]    0    0    0    0    0    0    0    0    0     0     0    11   190   253    70     0     0     0     0     0     0     0     0     0     0     0     0     0
[14,]    0    0    0    0    0    0    0    0    0     0     0     0    35   241   225   160   108     1     0     0     0     0     0     0     0     0     0     0
[15,]    0    0    0    0    0    0    0    0    0     0     0     0     0    81   240   253   253   119    25     0     0     0     0     0     0     0     0     0
[16,]    0    0    0    0    0    0    0    0    0     0     0     0     0     0    45   186   253   253   150    27     0     0     0     0     0     0     0     0
[17,]    0    0    0    0    0    0    0    0    0     0     0     0     0     0     0    16    93   252   253   187     0     0     0     0     0     0     0     0
[18,]    0    0    0    0    0    0    0    0    0     0     0     0     0     0     0     0     0   249   253   249    64     0     0     0     0     0     0     0
[19,]    0    0    0    0    0    0    0    0    0     0     0     0     0     0    46   130   183   253   253   207     2     0     0     0     0     0     0     0
[20,]    0    0    0    0    0    0    0    0    0     0     0     0    39   148   229   253   253   253   250   182     0     0     0     0     0     0     0     0
[21,]    0    0    0    0    0    0    0    0    0     0    24   114   221   253   253   253   253   201    78     0     0     0     0     0     0     0     0     0
[22,]    0    0    0    0    0    0    0    0   23    66   213   253   253   253   253   198    81     2     0     0     0     0     0     0     0     0     0     0
[23,]    0    0    0    0    0    0   18  171  219   253   253   253   253   195    80     9     0     0     0     0     0     0     0     0     0     0     0     0
[24,]    0    0    0    0   55  172  226  253  253   253   253   244   133    11     0     0     0     0     0     0     0     0     0     0     0     0     0     0
[25,]    0    0    0    0  136  253  253  253  212   135   132    16     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0
[26,]    0    0    0    0    0    0    0    0    0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0
[27,]    0    0    0    0    0    0    0    0    0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0
[28,]    0    0    0    0    0    0    0    0    0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0     0

Lets plot the first digit and compare to the above matrix

plot(as.raster(digit, max = 255))

Now lets check out the first 100 digits

par(mfrow = c(10, 10), mar = c(0,0,0,0))
for (i in 1:100) {
  plot(as.raster(mnist$train$x[i,,], max = 255))
}

Prepare Data

When we work with keras:

• training and test sets need to be independent
• features and labels (aka target, response) need to be independent
• use %<-% for object unpacking (see ?zeallot::%<-%)

c(c(train_images, train_labels), c(test_images, test_labels)) %<-% mnist

# the above is the same as
# train_images <- mnist$train$x
# train_labels <- mnist$train$y
# test_images <- mnist$test$x
# test_labels <- mnist$test$y

Shape into proper tensor form

The shape of our data is dependent on the type of DL model we are training. MLPs require our data to be in a 2D tensor (aka matrix); however, our data are currently in a 3D tensor.

We can reshape our tensor from 3D to 2D. Much like a matrix can be flattened to a vector:

m <- matrix(1:9, ncol = 3)
m

# flattened matrix
as.vector(m)

We can reshape a 3D array to a 2D array with array_reshape()

# reshape 3D tensor (aka array) to a 2D tensor (aka matrix)
train_images <- array_reshape(train_images, c(60000, 28 * 28))
test_images <- array_reshape(test_images, c(10000, 28 * 28))

# our training data is now a matrix with 60K observations and
# 784 features (28 pixels x 28 pixels = 784)
str(train_images)

 int [1:60000, 1:784] 0 0 0 0 0 0 0 0 0 0 ...

Since we are dealing with a multi-classification problem where the target ranges from 0-9, we’ll reformat with to_categorical().

Note: column 1 refers to the digit “0”, column 2 refers to the digit “1”, etc.

# to_categorical(c(3,6,8, 9))

train_labels <- to_categorical(train_labels)
test_labels <- to_categorical(test_labels)

head(train_labels)

     [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
[1,]    0    0    0    0    0    1    0    0    0     0
[2,]    1    0    0    0    0    0    0    0    0     0
[3,]    0    0    0    0    1    0    0    0    0     0
[4,]    0    1    0    0    0    0    0    0    0     0
[5,]    0    0    0    0    0    0    0    0    0     1
[6,]    0    0    1    0    0    0    0    0    0     0

Stabilize learning by data scaling

When applying DL models, our feature values should not be relatively large compared to the randomized initial weights and all our features should take values in roughly the same range.

When features have large or widely varying values, large gradient updates can be triggered that will prevent the network from converging

Tips:

1. When all features have the same value range (i.e. images), we can standardize values between 0-1.

2. When features varying in range from one another (i.e. age, height, longitude) normalize each feature to have mean of 0 and standard deviation of 1 (?scale())

# all our features (pixels) range from 0-255
range(train_images)

# standardize train and test features
train_images <- train_images / 255
test_images <- test_images / 255

Randomize data

Although I know that this data is not ordered, we should always get in the habit of randomizing our data so that our train and validation datasets are properly represented.

obs <- nrow(train_images)
set.seed(123)
randomize <- sample(seq_len(obs), size = obs, replace = FALSE)
train_images <- train_images[randomize, ]
train_labels <- train_labels[randomize, ]

We’re finally ready to train some DL models!!

Balance Batch Size & Default Learning Rate

The learning rate is the most important hyperparameter to get right but before I tuning it, I like to find a batch size that balances:

• smoothness of the learning curve
• speed of training

Tips: start with…

• a default learning rate,
• a single hidden layer with units set to around mean(n_features + n_response_classes),
• the default batch size of 32,
• then test ranges of batch sizes [16, 32, 64, 128, 256, 512].

# get number of features
n_feat <- ncol(train_images)

# 1. Define model architecture
model <- keras_model_sequential() %>%
  layer_dense(units = 2^9, activation = 'relu', input_shape = n_feat) %>%
  layer_dense(units = 10, activation = 'softmax')

Metal device set to: Apple M1 Max

systemMemory: 64.00 GB
maxCacheSize: 24.00 GB

2022-08-04 08:34:53.403417: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:305] Could not identify NUMA node of platform GPU ID 0, defaulting to 0. Your kernel may not have been built with NUMA support.
2022-08-04 08:34:53.403909: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:271] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 0 MB memory) -> physical PluggableDevice (device: 0, name: METAL, pci bus id: <undefined>)

model

Model: "sequential"
____________________________________________________________________________________________
 Layer (type)                            Output Shape                         Param #       
============================================================================================
 dense_1 (Dense)                         (None, 512)                          401920        
 dense (Dense)                           (None, 10)                           5130          
============================================================================================
Total params: 407,050
Trainable params: 407,050
Non-trainable params: 0
____________________________________________________________________________________________

model_2_hidden <- keras_model_sequential() %>%
  layer_dense(units = 2^9, activation = 'relu', input_shape = n_feat) %>%
  layer_dense(units = 2^9, activation = 'relu') %>%
  layer_dense(units = 10, activation = 'softmax')
model_2_hidden

Model: "sequential_1"
____________________________________________________________________________________________
 Layer (type)                            Output Shape                         Param #       
============================================================================================
 dense_4 (Dense)                         (None, 512)                          401920        
 dense_3 (Dense)                         (None, 512)                          262656        
 dense_2 (Dense)                         (None, 10)                           5130          
============================================================================================
Total params: 669,706
Trainable params: 669,706
Non-trainable params: 0
____________________________________________________________________________________________

model_large_hidden <- keras_model_sequential() %>%
  layer_dense(units = 2^10, activation = 'relu', input_shape = n_feat) %>%
  layer_dense(units = 10, activation = 'softmax')
model_large_hidden

Model: "sequential_2"
____________________________________________________________________________________________
 Layer (type)                            Output Shape                         Param #       
============================================================================================
 dense_6 (Dense)                         (None, 1024)                         803840        
 dense_5 (Dense)                         (None, 10)                           10250         
============================================================================================
Total params: 814,090
Trainable params: 814,090
Non-trainable params: 0
____________________________________________________________________________________________

# 2. Define how our model is going to learn
model %>% compile(
  loss = "categorical_crossentropy",
  optimizer = "sgd",
  metrics = "accuracy"
)
model

Model: "sequential"
____________________________________________________________________________________________
 Layer (type)                            Output Shape                         Param #       
============================================================================================
 dense_1 (Dense)                         (None, 512)                          401920        
 dense (Dense)                           (None, 10)                           5130          
============================================================================================
Total params: 407,050
Trainable params: 407,050
Non-trainable params: 0
____________________________________________________________________________________________

summary(model)

Model: "sequential"
____________________________________________________________________________________________
 Layer (type)                            Output Shape                         Param #       
============================================================================================
 dense_1 (Dense)                         (None, 512)                          401920        
 dense (Dense)                           (None, 10)                           5130          
============================================================================================
Total params: 407,050
Trainable params: 407,050
Non-trainable params: 0
____________________________________________________________________________________________

Let’s train our model with the default batch size of 32.

# 3. Train our model
history <- model %>% fit(
  train_images, train_labels,
  validation_split = 0.2
  )

2022-08-04 09:41:56.938978: W tensorflow/core/platform/profile_utils/cpu_utils.cc:128] Failed to get CPU frequency: 0 Hz

Epoch 1/10

2022-08-04 09:41:57.126302: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:113] Plugin optimizer for device_type GPU is enabled.

   1/1500 [..............................] - ETA: 48:41 - loss: 2.3579 - accuracy: 0.0625
   8/1500 [..............................] - ETA: 11s - loss: 2.3420 - accuracy: 0.1211  
  15/1500 [..............................] - ETA: 11s - loss: 2.2922 - accuracy: 0.1542
  22/1500 [..............................] - ETA: 11s - loss: 2.2405 - accuracy: 0.2060
  28/1500 [..............................] - ETA: 11s - loss: 2.2023 - accuracy: 0.2455
  35/1500 [..............................] - ETA: 11s - loss: 2.1697 - accuracy: 0.2804
  42/1500 [..............................] - ETA: 11s - loss: 2.1367 - accuracy: 0.3222
  49/1500 [..............................] - ETA: 11s - loss: 2.1006 - accuracy: 0.3597
  56/1500 [>.............................] - ETA: 11s - loss: 2.0667 - accuracy: 0.3862
  63/1500 [>.............................] - ETA: 11s - loss: 2.0372 - accuracy: 0.4058
  69/1500 [>.............................] - ETA: 11s - loss: 2.0083 - accuracy: 0.4316
  76/1500 [>.............................] - ETA: 11s - loss: 1.9737 - accuracy: 0.4560
  83/1500 [>.............................] - ETA: 11s - loss: 1.9389 - accuracy: 0.4774
  90/1500 [>.............................] - ETA: 11s - loss: 1.9093 - accuracy: 0.4965
  97/1500 [>.............................] - ETA: 11s - loss: 1.8824 - accuracy: 0.5106
 104/1500 [=>............................] - ETA: 11s - loss: 1.8527 - accuracy: 0.5249
 111/1500 [=>............................] - ETA: 10s - loss: 1.8230 - accuracy: 0.5400
 118/1500 [=>............................] - ETA: 10s - loss: 1.7986 - accuracy: 0.5498
 125/1500 [=>............................] - ETA: 10s - loss: 1.7738 - accuracy: 0.5595
 131/1500 [=>............................] - ETA: 10s - loss: 1.7525 - accuracy: 0.5677
 138/1500 [=>............................] - ETA: 10s - loss: 1.7264 - accuracy: 0.5777
 145/1500 [=>............................] - ETA: 10s - loss: 1.7043 - accuracy: 0.5851
 152/1500 [==>...........................] - ETA: 10s - loss: 1.6791 - accuracy: 0.5940
 157/1500 [==>...........................] - ETA: 10s - loss: 1.6645 - accuracy: 0.5995
 164/1500 [==>...........................] - ETA: 10s - loss: 1.6462 - accuracy: 0.6071
 171/1500 [==>...........................] - ETA: 10s - loss: 1.6251 - accuracy: 0.6148
 178/1500 [==>...........................] - ETA: 10s - loss: 1.6043 - accuracy: 0.6225
 185/1500 [==>...........................] - ETA: 10s - loss: 1.5849 - accuracy: 0.6280
 192/1500 [==>...........................] - ETA: 10s - loss: 1.5661 - accuracy: 0.6351
 199/1500 [==>...........................] - ETA: 10s - loss: 1.5472 - accuracy: 0.6410
 206/1500 [===>..........................] - ETA: 10s - loss: 1.5292 - accuracy: 0.6464
 213/1500 [===>..........................] - ETA: 10s - loss: 1.5106 - accuracy: 0.6517
 219/1500 [===>..........................] - ETA: 10s - loss: 1.4945 - accuracy: 0.6568
 226/1500 [===>..........................] - ETA: 10s - loss: 1.4784 - accuracy: 0.6616
 233/1500 [===>..........................] - ETA: 9s - loss: 1.4616 - accuracy: 0.6671 
 240/1500 [===>..........................] - ETA: 9s - loss: 1.4450 - accuracy: 0.6719
 246/1500 [===>..........................] - ETA: 9s - loss: 1.4318 - accuracy: 0.6757
 253/1500 [====>.........................] - ETA: 9s - loss: 1.4159 - accuracy: 0.6805
 260/1500 [====>.........................] - ETA: 9s - loss: 1.4018 - accuracy: 0.6837
 267/1500 [====>.........................] - ETA: 9s - loss: 1.3895 - accuracy: 0.6871
 274/1500 [====>.........................] - ETA: 9s - loss: 1.3762 - accuracy: 0.6894
 281/1500 [====>.........................] - ETA: 9s - loss: 1.3617 - accuracy: 0.6938
 288/1500 [====>.........................] - ETA: 9s - loss: 1.3468 - accuracy: 0.6979
 295/1500 [====>.........................] - ETA: 9s - loss: 1.3335 - accuracy: 0.7014
 302/1500 [=====>........................] - ETA: 9s - loss: 1.3214 - accuracy: 0.7048
 309/1500 [=====>........................] - ETA: 9s - loss: 1.3085 - accuracy: 0.7088
 316/1500 [=====>........................] - ETA: 9s - loss: 1.2965 - accuracy: 0.7119
 323/1500 [=====>........................] - ETA: 9s - loss: 1.2852 - accuracy: 0.7152
 329/1500 [=====>........................] - ETA: 9s - loss: 1.2749 - accuracy: 0.7180
 336/1500 [=====>........................] - ETA: 9s - loss: 1.2645 - accuracy: 0.7208
 343/1500 [=====>........................] - ETA: 9s - loss: 1.2536 - accuracy: 0.7234
 350/1500 [======>.......................] - ETA: 9s - loss: 1.2436 - accuracy: 0.7254
 357/1500 [======>.......................] - ETA: 9s - loss: 1.2342 - accuracy: 0.7274
 364/1500 [======>.......................] - ETA: 9s - loss: 1.2239 - accuracy: 0.7299
 371/1500 [======>.......................] - ETA: 8s - loss: 1.2132 - accuracy: 0.7326
 378/1500 [======>.......................] - ETA: 8s - loss: 1.2034 - accuracy: 0.7344
 385/1500 [======>.......................] - ETA: 8s - loss: 1.1943 - accuracy: 0.7367
 391/1500 [======>.......................] - ETA: 8s - loss: 1.1861 - accuracy: 0.7387
 398/1500 [======>.......................] - ETA: 8s - loss: 1.1792 - accuracy: 0.7401
 405/1500 [=======>......................] - ETA: 8s - loss: 1.1705 - accuracy: 0.7418
 412/1500 [=======>......................] - ETA: 8s - loss: 1.1626 - accuracy: 0.7432
 419/1500 [=======>......................] - ETA: 8s - loss: 1.1539 - accuracy: 0.7450
 426/1500 [=======>......................] - ETA: 8s - loss: 1.1456 - accuracy: 0.7468
 433/1500 [=======>......................] - ETA: 8s - loss: 1.1364 - accuracy: 0.7488
 440/1500 [=======>......................] - ETA: 8s - loss: 1.1281 - accuracy: 0.7509
 447/1500 [=======>......................] - ETA: 8s - loss: 1.1202 - accuracy: 0.7531
 454/1500 [========>.....................] - ETA: 8s - loss: 1.1124 - accuracy: 0.7550
 461/1500 [========>.....................] - ETA: 8s - loss: 1.1061 - accuracy: 0.7560
 468/1500 [========>.....................] - ETA: 8s - loss: 1.0978 - accuracy: 0.7578
 475/1500 [========>.....................] - ETA: 8s - loss: 1.0904 - accuracy: 0.7595
 481/1500 [========>.....................] - ETA: 8s - loss: 1.0841 - accuracy: 0.7607
 488/1500 [========>.....................] - ETA: 8s - loss: 1.0780 - accuracy: 0.7618
 495/1500 [========>.....................] - ETA: 7s - loss: 1.0708 - accuracy: 0.7636
 502/1500 [=========>....................] - ETA: 7s - loss: 1.0642 - accuracy: 0.7649
 509/1500 [=========>....................] - ETA: 7s - loss: 1.0582 - accuracy: 0.7660
 516/1500 [=========>....................] - ETA: 7s - loss: 1.0515 - accuracy: 0.7676
 523/1500 [=========>....................] - ETA: 7s - loss: 1.0457 - accuracy: 0.7684
 530/1500 [=========>....................] - ETA: 7s - loss: 1.0393 - accuracy: 0.7700
 537/1500 [=========>....................] - ETA: 7s - loss: 1.0323 - accuracy: 0.7716
 544/1500 [=========>....................] - ETA: 7s - loss: 1.0262 - accuracy: 0.7728
 551/1500 [==========>...................] - ETA: 7s - loss: 1.0199 - accuracy: 0.7745
 558/1500 [==========>...................] - ETA: 7s - loss: 1.0143 - accuracy: 0.7755
 564/1500 [==========>...................] - ETA: 7s - loss: 1.0100 - accuracy: 0.7764
 571/1500 [==========>...................] - ETA: 7s - loss: 1.0049 - accuracy: 0.7773
 578/1500 [==========>...................] - ETA: 7s - loss: 0.9993 - accuracy: 0.7783
 585/1500 [==========>...................] - ETA: 7s - loss: 0.9944 - accuracy: 0.7790
 592/1500 [==========>...................] - ETA: 7s - loss: 0.9894 - accuracy: 0.7801
 599/1500 [==========>...................] - ETA: 7s - loss: 0.9835 - accuracy: 0.7812
 606/1500 [===========>..................] - ETA: 7s - loss: 0.9788 - accuracy: 0.7817
 613/1500 [===========>..................] - ETA: 7s - loss: 0.9743 - accuracy: 0.7827
 620/1500 [===========>..................] - ETA: 6s - loss: 0.9700 - accuracy: 0.7836
 627/1500 [===========>..................] - ETA: 6s - loss: 0.9648 - accuracy: 0.7848
 634/1500 [===========>..................] - ETA: 6s - loss: 0.9605 - accuracy: 0.7854
 641/1500 [===========>..................] - ETA: 6s - loss: 0.9561 - accuracy: 0.7862
 648/1500 [===========>..................] - ETA: 6s - loss: 0.9522 - accuracy: 0.7867
 655/1500 [============>.................] - ETA: 6s - loss: 0.9483 - accuracy: 0.7871
 661/1500 [============>.................] - ETA: 6s - loss: 0.9447 - accuracy: 0.7876
 668/1500 [============>.................] - ETA: 6s - loss: 0.9408 - accuracy: 0.7884
 675/1500 [============>.................] - ETA: 6s - loss: 0.9360 - accuracy: 0.7895
 682/1500 [============>.................] - ETA: 6s - loss: 0.9322 - accuracy: 0.7899
 688/1500 [============>.................] - ETA: 6s - loss: 0.9280 - accuracy: 0.7910
 695/1500 [============>.................] - ETA: 6s - loss: 0.9237 - accuracy: 0.7917
 701/1500 [=============>................] - ETA: 6s - loss: 0.9195 - accuracy: 0.7928
 708/1500 [=============>................] - ETA: 6s - loss: 0.9153 - accuracy: 0.7938
 715/1500 [=============>................] - ETA: 6s - loss: 0.9114 - accuracy: 0.7948
 722/1500 [=============>................] - ETA: 6s - loss: 0.9074 - accuracy: 0.7956
 729/1500 [=============>................] - ETA: 6s - loss: 0.9028 - accuracy: 0.7964
 736/1500 [=============>................] - ETA: 6s - loss: 0.8984 - accuracy: 0.7974
 743/1500 [=============>................] - ETA: 5s - loss: 0.8940 - accuracy: 0.7984
 750/1500 [==============>...............] - ETA: 5s - loss: 0.8893 - accuracy: 0.7994
 757/1500 [==============>...............] - ETA: 5s - loss: 0.8861 - accuracy: 0.7999
 764/1500 [==============>...............] - ETA: 5s - loss: 0.8830 - accuracy: 0.8002
 771/1500 [==============>...............] - ETA: 5s - loss: 0.8795 - accuracy: 0.8009
 778/1500 [==============>...............] - ETA: 5s - loss: 0.8759 - accuracy: 0.8017
 785/1500 [==============>...............] - ETA: 5s - loss: 0.8724 - accuracy: 0.8023
 792/1500 [==============>...............] - ETA: 5s - loss: 0.8692 - accuracy: 0.8031
 799/1500 [==============>...............] - ETA: 5s - loss: 0.8657 - accuracy: 0.8038
 806/1500 [===============>..............] - ETA: 5s - loss: 0.8627 - accuracy: 0.8045
 813/1500 [===============>..............] - ETA: 5s - loss: 0.8592 - accuracy: 0.8054
 821/1500 [===============>..............] - ETA: 5s - loss: 0.8547 - accuracy: 0.8063
 829/1500 [===============>..............] - ETA: 5s - loss: 0.8509 - accuracy: 0.8071
 837/1500 [===============>..............] - ETA: 5s - loss: 0.8474 - accuracy: 0.8078
 844/1500 [===============>..............] - ETA: 5s - loss: 0.8448 - accuracy: 0.8083
 852/1500 [================>.............] - ETA: 5s - loss: 0.8410 - accuracy: 0.8092
 859/1500 [================>.............] - ETA: 5s - loss: 0.8375 - accuracy: 0.8101
 867/1500 [================>.............] - ETA: 4s - loss: 0.8344 - accuracy: 0.8106
 874/1500 [================>.............] - ETA: 4s - loss: 0.8320 - accuracy: 0.8109
 881/1500 [================>.............] - ETA: 4s - loss: 0.8295 - accuracy: 0.8114
 889/1500 [================>.............] - ETA: 4s - loss: 0.8274 - accuracy: 0.8114
 896/1500 [================>.............] - ETA: 4s - loss: 0.8246 - accuracy: 0.8119
 904/1500 [=================>............] - ETA: 4s - loss: 0.8215 - accuracy: 0.8123
 911/1500 [=================>............] - ETA: 4s - loss: 0.8190 - accuracy: 0.8128
 918/1500 [=================>............] - ETA: 4s - loss: 0.8163 - accuracy: 0.8132
 925/1500 [=================>............] - ETA: 4s - loss: 0.8136 - accuracy: 0.8138
 932/1500 [=================>............] - ETA: 4s - loss: 0.8106 - accuracy: 0.8145
 937/1500 [=================>............] - ETA: 4s - loss: 0.8089 - accuracy: 0.8148
 944/1500 [=================>............] - ETA: 4s - loss: 0.8063 - accuracy: 0.8154
 951/1500 [==================>...........] - ETA: 4s - loss: 0.8035 - accuracy: 0.8160
 958/1500 [==================>...........] - ETA: 4s - loss: 0.8011 - accuracy: 0.8164
 965/1500 [==================>...........] - ETA: 4s - loss: 0.7984 - accuracy: 0.8170
 972/1500 [==================>...........] - ETA: 4s - loss: 0.7957 - accuracy: 0.8176
 979/1500 [==================>...........] - ETA: 4s - loss: 0.7934 - accuracy: 0.8181
 986/1500 [==================>...........] - ETA: 4s - loss: 0.7912 - accuracy: 0.8185
 993/1500 [==================>...........] - ETA: 3s - loss: 0.7892 - accuracy: 0.8189
1000/1500 [===================>..........] - ETA: 3s - loss: 0.7869 - accuracy: 0.8194
1008/1500 [===================>..........] - ETA: 3s - loss: 0.7847 - accuracy: 0.8198
1015/1500 [===================>..........] - ETA: 3s - loss: 0.7824 - accuracy: 0.8199
1022/1500 [===================>..........] - ETA: 3s - loss: 0.7799 - accuracy: 0.8204
1029/1500 [===================>..........] - ETA: 3s - loss: 0.7781 - accuracy: 0.8206
1036/1500 [===================>..........] - ETA: 3s - loss: 0.7762 - accuracy: 0.8210
1040/1500 [===================>..........] - ETA: 3s - loss: 0.7747 - accuracy: 0.8215
1045/1500 [===================>..........] - ETA: 3s - loss: 0.7733 - accuracy: 0.8218
1052/1500 [====================>.........] - ETA: 3s - loss: 0.7706 - accuracy: 0.8224
1059/1500 [====================>.........] - ETA: 3s - loss: 0.7680 - accuracy: 0.8229
1066/1500 [====================>.........] - ETA: 3s - loss: 0.7654 - accuracy: 0.8235
1072/1500 [====================>.........] - ETA: 3s - loss: 0.7634 - accuracy: 0.8240
1079/1500 [====================>.........] - ETA: 3s - loss: 0.7615 - accuracy: 0.8243
1086/1500 [====================>.........] - ETA: 3s - loss: 0.7593 - accuracy: 0.8247
1093/1500 [====================>.........] - ETA: 3s - loss: 0.7573 - accuracy: 0.8250
1100/1500 [=====================>........] - ETA: 3s - loss: 0.7550 - accuracy: 0.8255
1107/1500 [=====================>........] - ETA: 3s - loss: 0.7532 - accuracy: 0.8258
1113/1500 [=====================>........] - ETA: 3s - loss: 0.7514 - accuracy: 0.8261
1120/1500 [=====================>........] - ETA: 2s - loss: 0.7487 - accuracy: 0.8267
1127/1500 [=====================>........] - ETA: 2s - loss: 0.7469 - accuracy: 0.8269
1134/1500 [=====================>........] - ETA: 2s - loss: 0.7446 - accuracy: 0.8275
1140/1500 [=====================>........] - ETA: 2s - loss: 0.7431 - accuracy: 0.8278
1146/1500 [=====================>........] - ETA: 2s - loss: 0.7411 - accuracy: 0.8282
1153/1500 [======================>.......] - ETA: 2s - loss: 0.7393 - accuracy: 0.8285
1160/1500 [======================>.......] - ETA: 2s - loss: 0.7377 - accuracy: 0.8288
1166/1500 [======================>.......] - ETA: 2s - loss: 0.7359 - accuracy: 0.8291
1172/1500 [======================>.......] - ETA: 2s - loss: 0.7340 - accuracy: 0.8294
1179/1500 [======================>.......] - ETA: 2s - loss: 0.7323 - accuracy: 0.8297
1186/1500 [======================>.......] - ETA: 2s - loss: 0.7309 - accuracy: 0.8299
1193/1500 [======================>.......] - ETA: 2s - loss: 0.7299 - accuracy: 0.8301
1200/1500 [=======================>......] - ETA: 2s - loss: 0.7279 - accuracy: 0.8305
1207/1500 [=======================>......] - ETA: 2s - loss: 0.7258 - accuracy: 0.8309
1214/1500 [=======================>......] - ETA: 2s - loss: 0.7242 - accuracy: 0.8312
1221/1500 [=======================>......] - ETA: 2s - loss: 0.7229 - accuracy: 0.8314
1228/1500 [=======================>......] - ETA: 2s - loss: 0.7209 - accuracy: 0.8317
1235/1500 [=======================>......] - ETA: 2s - loss: 0.7192 - accuracy: 0.8321
1242/1500 [=======================>......] - ETA: 2s - loss: 0.7173 - accuracy: 0.8324
1249/1500 [=======================>......] - ETA: 1s - loss: 0.7155 - accuracy: 0.8329
1256/1500 [========================>.....] - ETA: 1s - loss: 0.7141 - accuracy: 0.8332
1263/1500 [========================>.....] - ETA: 1s - loss: 0.7122 - accuracy: 0.8337
1266/1500 [========================>.....] - ETA: 1s - loss: 0.7115 - accuracy: 0.8338
1269/1500 [========================>.....] - ETA: 1s - loss: 0.7107 - accuracy: 0.8340
1276/1500 [========================>.....] - ETA: 1s - loss: 0.7088 - accuracy: 0.8343
1282/1500 [========================>.....] - ETA: 1s - loss: 0.7073 - accuracy: 0.8348
1289/1500 [========================>.....] - ETA: 1s - loss: 0.7054 - accuracy: 0.8351
1296/1500 [========================>.....] - ETA: 1s - loss: 0.7035 - accuracy: 0.8355
1303/1500 [=========================>....] - ETA: 1s - loss: 0.7021 - accuracy: 0.8357
1310/1500 [=========================>....] - ETA: 1s - loss: 0.7007 - accuracy: 0.8359
1316/1500 [=========================>....] - ETA: 1s - loss: 0.6996 - accuracy: 0.8361
1323/1500 [=========================>....] - ETA: 1s - loss: 0.6976 - accuracy: 0.8365
1329/1500 [=========================>....] - ETA: 1s - loss: 0.6961 - accuracy: 0.8368
1336/1500 [=========================>....] - ETA: 1s - loss: 0.6947 - accuracy: 0.8370
1342/1500 [=========================>....] - ETA: 1s - loss: 0.6937 - accuracy: 0.8372
1348/1500 [=========================>....] - ETA: 1s - loss: 0.6920 - accuracy: 0.8375
1355/1500 [==========================>...] - ETA: 1s - loss: 0.6907 - accuracy: 0.8378
1361/1500 [==========================>...] - ETA: 1s - loss: 0.6892 - accuracy: 0.8382
1368/1500 [==========================>...] - ETA: 1s - loss: 0.6877 - accuracy: 0.8385
1375/1500 [==========================>...] - ETA: 0s - loss: 0.6862 - accuracy: 0.8388
1382/1500 [==========================>...] - ETA: 0s - loss: 0.6845 - accuracy: 0.8391
1388/1500 [==========================>...] - ETA: 0s - loss: 0.6834 - accuracy: 0.8392
1395/1500 [==========================>...] - ETA: 0s - loss: 0.6819 - accuracy: 0.8394
1402/1500 [===========================>..] - ETA: 0s - loss: 0.6805 - accuracy: 0.8398
1409/1500 [===========================>..] - ETA: 0s - loss: 0.6789 - accuracy: 0.8400
1415/1500 [===========================>..] - ETA: 0s - loss: 0.6780 - accuracy: 0.8402
1422/1500 [===========================>..] - ETA: 0s - loss: 0.6770 - accuracy: 0.8404
1429/1500 [===========================>..] - ETA: 0s - loss: 0.6755 - accuracy: 0.8407
1436/1500 [===========================>..] - ETA: 0s - loss: 0.6740 - accuracy: 0.8409
1443/1500 [===========================>..] - ETA: 0s - loss: 0.6731 - accuracy: 0.8410
1450/1500 [============================>.] - ETA: 0s - loss: 0.6715 - accuracy: 0.8414
1457/1500 [============================>.] - ETA: 0s - loss: 0.6704 - accuracy: 0.8416
1464/1500 [============================>.] - ETA: 0s - loss: 0.6694 - accuracy: 0.8416
1470/1500 [============================>.] - ETA: 0s - loss: 0.6685 - accuracy: 0.8418
1477/1500 [============================>.] - ETA: 0s - loss: 0.6670 - accuracy: 0.8422
1484/1500 [============================>.] - ETA: 0s - loss: 0.6659 - accuracy: 0.8422
1491/1500 [============================>.] - ETA: 0s - loss: 0.6648 - accuracy: 0.8424
1498/1500 [============================>.] - ETA: 0s - loss: 0.6641 - accuracy: 0.8425
1500/1500 [==============================] - 14s 8ms/step - loss: 0.6637 - accuracy: 0.8426

2022-08-04 09:42:10.964963: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:113] Plugin optimizer for device_type GPU is enabled.


1500/1500 [==============================] - 17s 10ms/step - loss: 0.6637 - accuracy: 0.8426 - val_loss: 0.3783 - val_accuracy: 0.8972
Epoch 2/10

   1/1500 [..............................] - ETA: 34s - loss: 0.3856 - accuracy: 0.9375
   7/1500 [..............................] - ETA: 12s - loss: 0.3854 - accuracy: 0.9107
  13/1500 [..............................] - ETA: 13s - loss: 0.3660 - accuracy: 0.9183
  19/1500 [..............................] - ETA: 13s - loss: 0.3562 - accuracy: 0.9161
  25/1500 [..............................] - ETA: 13s - loss: 0.3771 - accuracy: 0.9112
  32/1500 [..............................] - ETA: 12s - loss: 0.3708 - accuracy: 0.9092
  39/1500 [..............................] - ETA: 12s - loss: 0.3700 - accuracy: 0.9095
  47/1500 [..............................] - ETA: 11s - loss: 0.3767 - accuracy: 0.9049
  54/1500 [>.............................] - ETA: 11s - loss: 0.3668 - accuracy: 0.9074
  61/1500 [>.............................] - ETA: 11s - loss: 0.3775 - accuracy: 0.9032
  68/1500 [>.............................] - ETA: 11s - loss: 0.3770 - accuracy: 0.9021
  75/1500 [>.............................] - ETA: 11s - loss: 0.3727 - accuracy: 0.9046
  82/1500 [>.............................] - ETA: 11s - loss: 0.3711 - accuracy: 0.9043
  88/1500 [>.............................] - ETA: 11s - loss: 0.3705 - accuracy: 0.9052
  94/1500 [>.............................] - ETA: 11s - loss: 0.3727 - accuracy: 0.9043
 100/1500 [=>............................] - ETA: 11s - loss: 0.3712 - accuracy: 0.9041
 107/1500 [=>............................] - ETA: 11s - loss: 0.3682 - accuracy: 0.9048
 114/1500 [=>............................] - ETA: 11s - loss: 0.3632 - accuracy: 0.9060
 121/1500 [=>............................] - ETA: 11s - loss: 0.3608 - accuracy: 0.9068
 128/1500 [=>............................] - ETA: 11s - loss: 0.3653 - accuracy: 0.9058
 134/1500 [=>............................] - ETA: 11s - loss: 0.3653 - accuracy: 0.9056
 141/1500 [=>............................] - ETA: 11s - loss: 0.3693 - accuracy: 0.9051
 148/1500 [=>............................] - ETA: 10s - loss: 0.3703 - accuracy: 0.9050
 155/1500 [==>...........................] - ETA: 10s - loss: 0.3724 - accuracy: 0.9044
 162/1500 [==>...........................] - ETA: 10s - loss: 0.3723 - accuracy: 0.9045
 169/1500 [==>...........................] - ETA: 10s - loss: 0.3753 - accuracy: 0.9027
 175/1500 [==>...........................] - ETA: 10s - loss: 0.3739 - accuracy: 0.9029
 182/1500 [==>...........................] - ETA: 10s - loss: 0.3740 - accuracy: 0.9025
 189/1500 [==>...........................] - ETA: 10s - loss: 0.3718 - accuracy: 0.9031
 196/1500 [==>...........................] - ETA: 10s - loss: 0.3764 - accuracy: 0.9005
 203/1500 [===>..........................] - ETA: 10s - loss: 0.3765 - accuracy: 0.9010
 210/1500 [===>..........................] - ETA: 10s - loss: 0.3793 - accuracy: 0.9004
 217/1500 [===>..........................] - ETA: 10s - loss: 0.3765 - accuracy: 0.9014
 224/1500 [===>..........................] - ETA: 10s - loss: 0.3766 - accuracy: 0.9000
 230/1500 [===>..........................] - ETA: 10s - loss: 0.3786 - accuracy: 0.8988
 237/1500 [===>..........................] - ETA: 10s - loss: 0.3769 - accuracy: 0.8982
 244/1500 [===>..........................] - ETA: 10s - loss: 0.3786 - accuracy: 0.8979
 251/1500 [====>.........................] - ETA: 10s - loss: 0.3774 - accuracy: 0.8985
 258/1500 [====>.........................] - ETA: 9s - loss: 0.3794 - accuracy: 0.8977 
 265/1500 [====>.........................] - ETA: 9s - loss: 0.3794 - accuracy: 0.8980
 272/1500 [====>.........................] - ETA: 9s - loss: 0.3801 - accuracy: 0.8973
 279/1500 [====>.........................] - ETA: 9s - loss: 0.3806 - accuracy: 0.8971
 286/1500 [====>.........................] - ETA: 9s - loss: 0.3786 - accuracy: 0.8973
 293/1500 [====>.........................] - ETA: 9s - loss: 0.3787 - accuracy: 0.8970
 300/1500 [=====>........................] - ETA: 9s - loss: 0.3798 - accuracy: 0.8960
 307/1500 [=====>........................] - ETA: 9s - loss: 0.3804 - accuracy: 0.8961
 314/1500 [=====>........................] - ETA: 9s - loss: 0.3819 - accuracy: 0.8959
 321/1500 [=====>........................] - ETA: 9s - loss: 0.3826 - accuracy: 0.8960
 328/1500 [=====>........................] - ETA: 9s - loss: 0.3821 - accuracy: 0.8965
 335/1500 [=====>........................] - ETA: 9s - loss: 0.3810 - accuracy: 0.8972
 342/1500 [=====>........................] - ETA: 9s - loss: 0.3791 - accuracy: 0.8981
 349/1500 [=====>........................] - ETA: 9s - loss: 0.3780 - accuracy: 0.8988
 356/1500 [======>.......................] - ETA: 9s - loss: 0.3780 - accuracy: 0.8990
 363/1500 [======>.......................] - ETA: 9s - loss: 0.3780 - accuracy: 0.8988
 370/1500 [======>.......................] - ETA: 9s - loss: 0.3767 - accuracy: 0.8992
 377/1500 [======>.......................] - ETA: 8s - loss: 0.3766 - accuracy: 0.8993
 383/1500 [======>.......................] - ETA: 8s - loss: 0.3766 - accuracy: 0.8990
 390/1500 [======>.......................] - ETA: 8s - loss: 0.3764 - accuracy: 0.8989
 394/1500 [======>.......................] - ETA: 8s - loss: 0.3762 - accuracy: 0.8991
 401/1500 [=======>......................] - ETA: 8s - loss: 0.3777 - accuracy: 0.8985
 406/1500 [=======>......................] - ETA: 8s - loss: 0.3780 - accuracy: 0.8985
 413/1500 [=======>......................] - ETA: 8s - loss: 0.3777 - accuracy: 0.8985
 420/1500 [=======>......................] - ETA: 8s - loss: 0.3777 - accuracy: 0.8985
 427/1500 [=======>......................] - ETA: 8s - loss: 0.3766 - accuracy: 0.8989
 434/1500 [=======>......................] - ETA: 8s - loss: 0.3769 - accuracy: 0.8989
 441/1500 [=======>......................] - ETA: 8s - loss: 0.3755 - accuracy: 0.8994
 448/1500 [=======>......................] - ETA: 8s - loss: 0.3753 - accuracy: 0.8995
 455/1500 [========>.....................] - ETA: 8s - loss: 0.3748 - accuracy: 0.8993
 462/1500 [========>.....................] - ETA: 8s - loss: 0.3740 - accuracy: 0.8998
 469/1500 [========>.....................] - ETA: 8s - loss: 0.3738 - accuracy: 0.8997
 476/1500 [========>.....................] - ETA: 8s - loss: 0.3736 - accuracy: 0.8994
 483/1500 [========>.....................] - ETA: 8s - loss: 0.3729 - accuracy: 0.8998
 490/1500 [========>.....................] - ETA: 8s - loss: 0.3716 - accuracy: 0.9003
 497/1500 [========>.....................] - ETA: 8s - loss: 0.3700 - accuracy: 0.9009
 504/1500 [=========>....................] - ETA: 7s - loss: 0.3701 - accuracy: 0.9009
 510/1500 [=========>....................] - ETA: 7s - loss: 0.3689 - accuracy: 0.9010
 516/1500 [=========>....................] - ETA: 7s - loss: 0.3707 - accuracy: 0.9003
 522/1500 [=========>....................] - ETA: 7s - loss: 0.3702 - accuracy: 0.9005
 529/1500 [=========>....................] - ETA: 7s - loss: 0.3699 - accuracy: 0.9008
 536/1500 [=========>....................] - ETA: 7s - loss: 0.3693 - accuracy: 0.9009
 542/1500 [=========>....................] - ETA: 7s - loss: 0.3697 - accuracy: 0.9008
 548/1500 [=========>....................] - ETA: 7s - loss: 0.3698 - accuracy: 0.9007
 554/1500 [==========>...................] - ETA: 7s - loss: 0.3708 - accuracy: 0.9004
 560/1500 [==========>...................] - ETA: 7s - loss: 0.3714 - accuracy: 0.9002
 566/1500 [==========>...................] - ETA: 7s - loss: 0.3723 - accuracy: 0.9000
 572/1500 [==========>...................] - ETA: 7s - loss: 0.3714 - accuracy: 0.9002
 578/1500 [==========>...................] - ETA: 7s - loss: 0.3714 - accuracy: 0.9001
 583/1500 [==========>...................] - ETA: 7s - loss: 0.3710 - accuracy: 0.8999
 588/1500 [==========>...................] - ETA: 7s - loss: 0.3708 - accuracy: 0.8999
 595/1500 [==========>...................] - ETA: 7s - loss: 0.3708 - accuracy: 0.8996
 602/1500 [===========>..................] - ETA: 7s - loss: 0.3702 - accuracy: 0.8999
 609/1500 [===========>..................] - ETA: 7s - loss: 0.3691 - accuracy: 0.9000
 615/1500 [===========>..................] - ETA: 7s - loss: 0.3696 - accuracy: 0.8998
 622/1500 [===========>..................] - ETA: 7s - loss: 0.3685 - accuracy: 0.9001
 629/1500 [===========>..................] - ETA: 7s - loss: 0.3694 - accuracy: 0.9001
 636/1500 [===========>..................] - ETA: 7s - loss: 0.3694 - accuracy: 0.9001
 641/1500 [===========>..................] - ETA: 7s - loss: 0.3700 - accuracy: 0.9000
 643/1500 [===========>..................] - ETA: 7s - loss: 0.3702 - accuracy: 0.8999
 649/1500 [===========>..................] - ETA: 7s - loss: 0.3705 - accuracy: 0.8997
 656/1500 [============>.................] - ETA: 6s - loss: 0.3698 - accuracy: 0.8999
 663/1500 [============>.................] - ETA: 6s - loss: 0.3695 - accuracy: 0.9000
 670/1500 [============>.................] - ETA: 6s - loss: 0.3692 - accuracy: 0.9002
 677/1500 [============>.................] - ETA: 6s - loss: 0.3687 - accuracy: 0.9004
 684/1500 [============>.................] - ETA: 6s - loss: 0.3681 - accuracy: 0.9004
 691/1500 [============>.................] - ETA: 6s - loss: 0.3678 - accuracy: 0.9003
 698/1500 [============>.................] - ETA: 6s - loss: 0.3675 - accuracy: 0.9005
 705/1500 [=============>................] - ETA: 6s - loss: 0.3668 - accuracy: 0.9008
 712/1500 [=============>................] - ETA: 6s - loss: 0.3661 - accuracy: 0.9009
 719/1500 [=============>................] - ETA: 6s - loss: 0.3657 - accuracy: 0.9010
 726/1500 [=============>................] - ETA: 6s - loss: 0.3655 - accuracy: 0.9010
 733/1500 [=============>................] - ETA: 6s - loss: 0.3655 - accuracy: 0.9008
 739/1500 [=============>................] - ETA: 6s - loss: 0.3657 - accuracy: 0.9007
 746/1500 [=============>................] - ETA: 6s - loss: 0.3651 - accuracy: 0.9009
 753/1500 [==============>...............] - ETA: 6s - loss: 0.3646 - accuracy: 0.9010
 760/1500 [==============>...............] - ETA: 6s - loss: 0.3645 - accuracy: 0.9011
 767/1500 [==============>...............] - ETA: 6s - loss: 0.3643 - accuracy: 0.9011
 774/1500 [==============>...............] - ETA: 5s - loss: 0.3641 - accuracy: 0.9011
 781/1500 [==============>...............] - ETA: 5s - loss: 0.3641 - accuracy: 0.9010
 788/1500 [==============>...............] - ETA: 5s - loss: 0.3651 - accuracy: 0.9006
 795/1500 [==============>...............] - ETA: 5s - loss: 0.3644 - accuracy: 0.9009
 802/1500 [===============>..............] - ETA: 5s - loss: 0.3645 - accuracy: 0.9009
 809/1500 [===============>..............] - ETA: 5s - loss: 0.3636 - accuracy: 0.9011
 816/1500 [===============>..............] - ETA: 5s - loss: 0.3635 - accuracy: 0.9013
 823/1500 [===============>..............] - ETA: 5s - loss: 0.3631 - accuracy: 0.9014
 830/1500 [===============>..............] - ETA: 5s - loss: 0.3625 - accuracy: 0.9016
 837/1500 [===============>..............] - ETA: 5s - loss: 0.3620 - accuracy: 0.9017
 843/1500 [===============>..............] - ETA: 5s - loss: 0.3615 - accuracy: 0.9017
 850/1500 [================>.............] - ETA: 5s - loss: 0.3606 - accuracy: 0.9020
 856/1500 [================>.............] - ETA: 5s - loss: 0.3606 - accuracy: 0.9022
 863/1500 [================>.............] - ETA: 5s - loss: 0.3605 - accuracy: 0.9021
 870/1500 [================>.............] - ETA: 5s - loss: 0.3599 - accuracy: 0.9023
 877/1500 [================>.............] - ETA: 5s - loss: 0.3597 - accuracy: 0.9024
 884/1500 [================>.............] - ETA: 5s - loss: 0.3593 - accuracy: 0.9023
 890/1500 [================>.............] - ETA: 5s - loss: 0.3592 - accuracy: 0.9023
 896/1500 [================>.............] - ETA: 4s - loss: 0.3592 - accuracy: 0.9023
 902/1500 [=================>............] - ETA: 4s - loss: 0.3588 - accuracy: 0.9024
 909/1500 [=================>............] - ETA: 4s - loss: 0.3587 - accuracy: 0.9024
 915/1500 [=================>............] - ETA: 4s - loss: 0.3582 - accuracy: 0.9026
 921/1500 [=================>............] - ETA: 4s - loss: 0.3575 - accuracy: 0.9030
 928/1500 [=================>............] - ETA: 4s - loss: 0.3574 - accuracy: 0.9031
 935/1500 [=================>............] - ETA: 4s - loss: 0.3581 - accuracy: 0.9032
 942/1500 [=================>............] - ETA: 4s - loss: 0.3578 - accuracy: 0.9033
 948/1500 [=================>............] - ETA: 4s - loss: 0.3574 - accuracy: 0.9034
 955/1500 [==================>...........] - ETA: 4s - loss: 0.3570 - accuracy: 0.9035
 962/1500 [==================>...........] - ETA: 4s - loss: 0.3572 - accuracy: 0.9034
 969/1500 [==================>...........] - ETA: 4s - loss: 0.3573 - accuracy: 0.9035
 976/1500 [==================>...........] - ETA: 4s - loss: 0.3576 - accuracy: 0.9033
 983/1500 [==================>...........] - ETA: 4s - loss: 0.3580 - accuracy: 0.9031
 990/1500 [==================>...........] - ETA: 4s - loss: 0.3576 - accuracy: 0.9033
 997/1500 [==================>...........] - ETA: 4s - loss: 0.3573 - accuracy: 0.9033
1004/1500 [===================>..........] - ETA: 4s - loss: 0.3576 - accuracy: 0.9033
1011/1500 [===================>..........] - ETA: 4s - loss: 0.3570 - accuracy: 0.9036
1018/1500 [===================>..........] - ETA: 3s - loss: 0.3569 - accuracy: 0.9036
1025/1500 [===================>..........] - ETA: 3s - loss: 0.3570 - accuracy: 0.9035
1032/1500 [===================>..........] - ETA: 3s - loss: 0.3575 - accuracy: 0.9033
1039/1500 [===================>..........] - ETA: 3s - loss: 0.3573 - accuracy: 0.9034
1045/1500 [===================>..........] - ETA: 3s - loss: 0.3574 - accuracy: 0.9034
1052/1500 [====================>.........] - ETA: 3s - loss: 0.3573 - accuracy: 0.9035
1059/1500 [====================>.........] - ETA: 3s - loss: 0.3570 - accuracy: 0.9035
1065/1500 [====================>.........] - ETA: 3s - loss: 0.3567 - accuracy: 0.9036
1072/1500 [====================>.........] - ETA: 3s - loss: 0.3561 - accuracy: 0.9037
1079/1500 [====================>.........] - ETA: 3s - loss: 0.3557 - accuracy: 0.9037
1086/1500 [====================>.........] - ETA: 3s - loss: 0.3559 - accuracy: 0.9037
1093/1500 [====================>.........] - ETA: 3s - loss: 0.3562 - accuracy: 0.9035
1096/1500 [====================>.........] - ETA: 3s - loss: 0.3564 - accuracy: 0.9034
1097/1500 [====================>.........] - ETA: 3s - loss: 0.3563 - accuracy: 0.9034
1104/1500 [=====================>........] - ETA: 3s - loss: 0.3559 - accuracy: 0.9035
1111/1500 [=====================>........] - ETA: 3s - loss: 0.3558 - accuracy: 0.9036
1118/1500 [=====================>........] - ETA: 3s - loss: 0.3557 - accuracy: 0.9037
1125/1500 [=====================>........] - ETA: 3s - loss: 0.3557 - accuracy: 0.9037
1132/1500 [=====================>........] - ETA: 3s - loss: 0.3558 - accuracy: 0.9036
1139/1500 [=====================>........] - ETA: 2s - loss: 0.3557 - accuracy: 0.9037
1146/1500 [=====================>........] - ETA: 2s - loss: 0.3549 - accuracy: 0.9040
1153/1500 [======================>.......] - ETA: 2s - loss: 0.3546 - accuracy: 0.9040
1160/1500 [======================>.......] - ETA: 2s - loss: 0.3542 - accuracy: 0.9041
1167/1500 [======================>.......] - ETA: 2s - loss: 0.3540 - accuracy: 0.9042
1174/1500 [======================>.......] - ETA: 2s - loss: 0.3538 - accuracy: 0.9042
1181/1500 [======================>.......] - ETA: 2s - loss: 0.3538 - accuracy: 0.9042
1188/1500 [======================>.......] - ETA: 2s - loss: 0.3542 - accuracy: 0.9041
1195/1500 [======================>.......] - ETA: 2s - loss: 0.3543 - accuracy: 0.9039
1202/1500 [=======================>......] - ETA: 2s - loss: 0.3540 - accuracy: 0.9040
1209/1500 [=======================>......] - ETA: 2s - loss: 0.3538 - accuracy: 0.9039
1217/1500 [=======================>......] - ETA: 2s - loss: 0.3532 - accuracy: 0.9041
1224/1500 [=======================>......] - ETA: 2s - loss: 0.3529 - accuracy: 0.9042
1232/1500 [=======================>......] - ETA: 2s - loss: 0.3533 - accuracy: 0.9039
1237/1500 [=======================>......] - ETA: 2s - loss: 0.3530 - accuracy: 0.9039
1244/1500 [=======================>......] - ETA: 2s - loss: 0.3526 - accuracy: 0.9039
1251/1500 [========================>.....] - ETA: 2s - loss: 0.3522 - accuracy: 0.9041
1256/1500 [========================>.....] - ETA: 1s - loss: 0.3518 - accuracy: 0.9042
1264/1500 [========================>.....] - ETA: 1s - loss: 0.3514 - accuracy: 0.9044
1271/1500 [========================>.....] - ETA: 1s - loss: 0.3509 - accuracy: 0.9044
1278/1500 [========================>.....] - ETA: 1s - loss: 0.3504 - accuracy: 0.9045
1285/1500 [========================>.....] - ETA: 1s - loss: 0.3506 - accuracy: 0.9045
1289/1500 [========================>.....] - ETA: 1s - loss: 0.3507 - accuracy: 0.9044
1296/1500 [========================>.....] - ETA: 1s - loss: 0.3503 - accuracy: 0.9044
1303/1500 [=========================>....] - ETA: 1s - loss: 0.3497 - accuracy: 0.9047
1310/1500 [=========================>....] - ETA: 1s - loss: 0.3498 - accuracy: 0.9046
1317/1500 [=========================>....] - ETA: 1s - loss: 0.3496 - accuracy: 0.9047
1323/1500 [=========================>....] - ETA: 1s - loss: 0.3495 - accuracy: 0.9048
1330/1500 [=========================>....] - ETA: 1s - loss: 0.3493 - accuracy: 0.9048
1337/1500 [=========================>....] - ETA: 1s - loss: 0.3490 - accuracy: 0.9049
1344/1500 [=========================>....] - ETA: 1s - loss: 0.3490 - accuracy: 0.9048
1351/1500 [==========================>...] - ETA: 1s - loss: 0.3485 - accuracy: 0.9049
1357/1500 [==========================>...] - ETA: 1s - loss: 0.3484 - accuracy: 0.9049
1364/1500 [==========================>...] - ETA: 1s - loss: 0.3479 - accuracy: 0.9051
1371/1500 [==========================>...] - ETA: 1s - loss: 0.3477 - accuracy: 0.9051
1378/1500 [==========================>...] - ETA: 0s - loss: 0.3475 - accuracy: 0.9052
1385/1500 [==========================>...] - ETA: 0s - loss: 0.3475 - accuracy: 0.9052
1392/1500 [==========================>...] - ETA: 0s - loss: 0.3473 - accuracy: 0.9053
1399/1500 [==========================>...] - ETA: 0s - loss: 0.3470 - accuracy: 0.9054
1406/1500 [===========================>..] - ETA: 0s - loss: 0.3473 - accuracy: 0.9052
1413/1500 [===========================>..] - ETA: 0s - loss: 0.3470 - accuracy: 0.9052
1419/1500 [===========================>..] - ETA: 0s - loss: 0.3468 - accuracy: 0.9053
1426/1500 [===========================>..] - ETA: 0s - loss: 0.3466 - accuracy: 0.9054
1433/1500 [===========================>..] - ETA: 0s - loss: 0.3463 - accuracy: 0.9055
1440/1500 [===========================>..] - ETA: 0s - loss: 0.3463 - accuracy: 0.9054
1447/1500 [===========================>..] - ETA: 0s - loss: 0.3467 - accuracy: 0.9053
1454/1500 [============================>.] - ETA: 0s - loss: 0.3464 - accuracy: 0.9055
1461/1500 [============================>.] - ETA: 0s - loss: 0.3463 - accuracy: 0.9054
1468/1500 [============================>.] - ETA: 0s - loss: 0.3459 - accuracy: 0.9055
1474/1500 [============================>.] - ETA: 0s - loss: 0.3457 - accuracy: 0.9056
1481/1500 [============================>.] - ETA: 0s - loss: 0.3456 - accuracy: 0.9055
1488/1500 [============================>.] - ETA: 0s - loss: 0.3454 - accuracy: 0.9055
1496/1500 [============================>.] - ETA: 0s - loss: 0.3453 - accuracy: 0.9055
1500/1500 [==============================] - 12s 8ms/step - loss: 0.3453 - accuracy: 0.9054

1500/1500 [==============================] - 15s 10ms/step - loss: 0.3453 - accuracy: 0.9054 - val_loss: 0.3081 - val_accuracy: 0.9118
Epoch 3/10

   1/1500 [..............................] - ETA: 25s - loss: 0.1507 - accuracy: 0.9688
   8/1500 [..............................] - ETA: 10s - loss: 0.3169 - accuracy: 0.9258
  15/1500 [..............................] - ETA: 10s - loss: 0.2927 - accuracy: 0.9188
  23/1500 [..............................] - ETA: 10s - loss: 0.2968 - accuracy: 0.9171
  30/1500 [..............................] - ETA: 10s - loss: 0.3148 - accuracy: 0.9135
  37/1500 [..............................] - ETA: 10s - loss: 0.3313 - accuracy: 0.9096
  42/1500 [..............................] - ETA: 11s - loss: 0.3273 - accuracy: 0.9115
  49/1500 [..............................] - ETA: 11s - loss: 0.3166 - accuracy: 0.9171
  56/1500 [>.............................] - ETA: 11s - loss: 0.3176 - accuracy: 0.9141
  63/1500 [>.............................] - ETA: 11s - loss: 0.3315 - accuracy: 0.9072
  70/1500 [>.............................] - ETA: 10s - loss: 0.3292 - accuracy: 0.9071
  78/1500 [>.............................] - ETA: 10s - loss: 0.3211 - accuracy: 0.9079
  85/1500 [>.............................] - ETA: 10s - loss: 0.3216 - accuracy: 0.9074
  92/1500 [>.............................] - ETA: 10s - loss: 0.3206 - accuracy: 0.9066
  99/1500 [>.............................] - ETA: 10s - loss: 0.3200 - accuracy: 0.9059
 106/1500 [=>............................] - ETA: 10s - loss: 0.3211 - accuracy: 0.9065
 113/1500 [=>............................] - ETA: 10s - loss: 0.3222 - accuracy: 0.9076
 121/1500 [=>............................] - ETA: 10s - loss: 0.3183 - accuracy: 0.9088
 128/1500 [=>............................] - ETA: 10s - loss: 0.3134 - accuracy: 0.9109
 135/1500 [=>............................] - ETA: 10s - loss: 0.3154 - accuracy: 0.9116
 141/1500 [=>............................] - ETA: 10s - loss: 0.3150 - accuracy: 0.9118
 148/1500 [=>............................] - ETA: 10s - loss: 0.3139 - accuracy: 0.9126
 155/1500 [==>...........................] - ETA: 10s - loss: 0.3125 - accuracy: 0.9129
 162/1500 [==>...........................] - ETA: 10s - loss: 0.3130 - accuracy: 0.9124
 169/1500 [==>...........................] - ETA: 10s - loss: 0.3112 - accuracy: 0.9129
 176/1500 [==>...........................] - ETA: 10s - loss: 0.3112 - accuracy: 0.9137
 183/1500 [==>...........................] - ETA: 9s - loss: 0.3101 - accuracy: 0.9138 
 190/1500 [==>...........................] - ETA: 9s - loss: 0.3118 - accuracy: 0.9133
 197/1500 [==>...........................] - ETA: 9s - loss: 0.3164 - accuracy: 0.9120
 204/1500 [===>..........................] - ETA: 9s - loss: 0.3178 - accuracy: 0.9113
 211/1500 [===>..........................] - ETA: 9s - loss: 0.3167 - accuracy: 0.9116
 218/1500 [===>..........................] - ETA: 9s - loss: 0.3160 - accuracy: 0.9117
 225/1500 [===>..........................] - ETA: 9s - loss: 0.3171 - accuracy: 0.9118
 232/1500 [===>..........................] - ETA: 9s - loss: 0.3171 - accuracy: 0.9118
 239/1500 [===>..........................] - ETA: 9s - loss: 0.3162 - accuracy: 0.9119
 243/1500 [===>..........................] - ETA: 9s - loss: 0.3155 - accuracy: 0.9119
 250/1500 [====>.........................] - ETA: 9s - loss: 0.3145 - accuracy: 0.9123
 257/1500 [====>.........................] - ETA: 9s - loss: 0.3147 - accuracy: 0.9118
 263/1500 [====>.........................] - ETA: 9s - loss: 0.3185 - accuracy: 0.9102
 270/1500 [====>.........................] - ETA: 9s - loss: 0.3218 - accuracy: 0.9088
 277/1500 [====>.........................] - ETA: 9s - loss: 0.3225 - accuracy: 0.9090
 284/1500 [====>.........................] - ETA: 9s - loss: 0.3221 - accuracy: 0.9087
 291/1500 [====>.........................] - ETA: 9s - loss: 0.3233 - accuracy: 0.9079
 298/1500 [====>.........................] - ETA: 9s - loss: 0.3207 - accuracy: 0.9087
 305/1500 [=====>........................] - ETA: 9s - loss: 0.3198 - accuracy: 0.9089
 312/1500 [=====>........................] - ETA: 9s - loss: 0.3186 - accuracy: 0.9090
 318/1500 [=====>........................] - ETA: 9s - loss: 0.3188 - accuracy: 0.9089
 325/1500 [=====>........................] - ETA: 9s - loss: 0.3156 - accuracy: 0.9101
 332/1500 [=====>........................] - ETA: 9s - loss: 0.3171 - accuracy: 0.9097
 339/1500 [=====>........................] - ETA: 9s - loss: 0.3158 - accuracy: 0.9102
 346/1500 [=====>........................] - ETA: 9s - loss: 0.3144 - accuracy: 0.9107
 353/1500 [======>.......................] - ETA: 9s - loss: 0.3147 - accuracy: 0.9102
 360/1500 [======>.......................] - ETA: 8s - loss: 0.3143 - accuracy: 0.9106
 366/1500 [======>.......................] - ETA: 8s - loss: 0.3131 - accuracy: 0.9110
 373/1500 [======>.......................] - ETA: 8s - loss: 0.3136 - accuracy: 0.9109
 380/1500 [======>.......................] - ETA: 8s - loss: 0.3167 - accuracy: 0.9104
 387/1500 [======>.......................] - ETA: 8s - loss: 0.3167 - accuracy: 0.9102
 394/1500 [======>.......................] - ETA: 8s - loss: 0.3177 - accuracy: 0.9102
 401/1500 [=======>......................] - ETA: 8s - loss: 0.3166 - accuracy: 0.9108
 408/1500 [=======>......................] - ETA: 8s - loss: 0.3148 - accuracy: 0.9112
 415/1500 [=======>......................] - ETA: 8s - loss: 0.3137 - accuracy: 0.9117
 422/1500 [=======>......................] - ETA: 8s - loss: 0.3152 - accuracy: 0.9118
 429/1500 [=======>......................] - ETA: 8s - loss: 0.3132 - accuracy: 0.9125
 436/1500 [=======>......................] - ETA: 8s - loss: 0.3130 - accuracy: 0.9125
 443/1500 [=======>......................] - ETA: 8s - loss: 0.3129 - accuracy: 0.9127
 450/1500 [========>.....................] - ETA: 8s - loss: 0.3125 - accuracy: 0.9129
 457/1500 [========>.....................] - ETA: 8s - loss: 0.3117 - accuracy: 0.9131
 462/1500 [========>.....................] - ETA: 8s - loss: 0.3105 - accuracy: 0.9134
 469/1500 [========>.....................] - ETA: 8s - loss: 0.3100 - accuracy: 0.9135
 476/1500 [========>.....................] - ETA: 8s - loss: 0.3088 - accuracy: 0.9141
 483/1500 [========>.....................] - ETA: 8s - loss: 0.3071 - accuracy: 0.9146
 490/1500 [========>.....................] - ETA: 7s - loss: 0.3065 - accuracy: 0.9149
 497/1500 [========>.....................] - ETA: 7s - loss: 0.3062 - accuracy: 0.9149
 504/1500 [=========>....................] - ETA: 7s - loss: 0.3072 - accuracy: 0.9148
 511/1500 [=========>....................] - ETA: 7s - loss: 0.3066 - accuracy: 0.9152
 518/1500 [=========>....................] - ETA: 7s - loss: 0.3063 - accuracy: 0.9154
 525/1500 [=========>....................] - ETA: 7s - loss: 0.3064 - accuracy: 0.9155
 532/1500 [=========>....................] - ETA: 7s - loss: 0.3072 - accuracy: 0.9152
 539/1500 [=========>....................] - ETA: 7s - loss: 0.3068 - accuracy: 0.9154
 546/1500 [=========>....................] - ETA: 7s - loss: 0.3078 - accuracy: 0.9152
 553/1500 [==========>...................] - ETA: 7s - loss: 0.3090 - accuracy: 0.9149
 560/1500 [==========>...................] - ETA: 7s - loss: 0.3094 - accuracy: 0.9148
 567/1500 [==========>...................] - ETA: 7s - loss: 0.3087 - accuracy: 0.9148
 574/1500 [==========>...................] - ETA: 7s - loss: 0.3095 - accuracy: 0.9145
 581/1500 [==========>...................] - ETA: 7s - loss: 0.3097 - accuracy: 0.9146
 588/1500 [==========>...................] - ETA: 7s - loss: 0.3093 - accuracy: 0.9148
 595/1500 [==========>...................] - ETA: 7s - loss: 0.3098 - accuracy: 0.9144
 602/1500 [===========>..................] - ETA: 7s - loss: 0.3094 - accuracy: 0.9146
 609/1500 [===========>..................] - ETA: 7s - loss: 0.3092 - accuracy: 0.9146
 615/1500 [===========>..................] - ETA: 7s - loss: 0.3101 - accuracy: 0.9143
 622/1500 [===========>..................] - ETA: 6s - loss: 0.3103 - accuracy: 0.9142
 629/1500 [===========>..................] - ETA: 6s - loss: 0.3100 - accuracy: 0.9143
 636/1500 [===========>..................] - ETA: 6s - loss: 0.3093 - accuracy: 0.9144
 643/1500 [===========>..................] - ETA: 6s - loss: 0.3097 - accuracy: 0.9141
 650/1500 [============>.................] - ETA: 6s - loss: 0.3094 - accuracy: 0.9145
 657/1500 [============>.................] - ETA: 6s - loss: 0.3093 - accuracy: 0.9145
 663/1500 [============>.................] - ETA: 6s - loss: 0.3098 - accuracy: 0.9144
 670/1500 [============>.................] - ETA: 6s - loss: 0.3086 - accuracy: 0.9147
 677/1500 [============>.................] - ETA: 6s - loss: 0.3080 - accuracy: 0.9150
 684/1500 [============>.................] - ETA: 6s - loss: 0.3083 - accuracy: 0.9148
 691/1500 [============>.................] - ETA: 6s - loss: 0.3081 - accuracy: 0.9148
 698/1500 [============>.................] - ETA: 6s - loss: 0.3079 - accuracy: 0.9147
 705/1500 [=============>................] - ETA: 6s - loss: 0.3070 - accuracy: 0.9148
 712/1500 [=============>................] - ETA: 6s - loss: 0.3057 - accuracy: 0.9152
 718/1500 [=============>................] - ETA: 6s - loss: 0.3056 - accuracy: 0.9151
 725/1500 [=============>................] - ETA: 6s - loss: 0.3058 - accuracy: 0.9150
 730/1500 [=============>................] - ETA: 6s - loss: 0.3060 - accuracy: 0.9150
 733/1500 [=============>................] - ETA: 6s - loss: 0.3056 - accuracy: 0.9151
 738/1500 [=============>................] - ETA: 6s - loss: 0.3055 - accuracy: 0.9151
 745/1500 [=============>................] - ETA: 6s - loss: 0.3056 - accuracy: 0.9150
 751/1500 [==============>...............] - ETA: 6s - loss: 0.3052 - accuracy: 0.9152
 758/1500 [==============>...............] - ETA: 5s - loss: 0.3059 - accuracy: 0.9150
 765/1500 [==============>...............] - ETA: 5s - loss: 0.3054 - accuracy: 0.9152
 772/1500 [==============>...............] - ETA: 5s - loss: 0.3053 - accuracy: 0.9152
 779/1500 [==============>...............] - ETA: 5s - loss: 0.3052 - accuracy: 0.9152
 786/1500 [==============>...............] - ETA: 5s - loss: 0.3048 - accuracy: 0.9153
 792/1500 [==============>...............] - ETA: 5s - loss: 0.3039 - accuracy: 0.9156
 798/1500 [==============>...............] - ETA: 5s - loss: 0.3039 - accuracy: 0.9156
 805/1500 [===============>..............] - ETA: 5s - loss: 0.3045 - accuracy: 0.9153
 813/1500 [===============>..............] - ETA: 5s - loss: 0.3052 - accuracy: 0.9150
 820/1500 [===============>..............] - ETA: 5s - loss: 0.3045 - accuracy: 0.9152
 827/1500 [===============>..............] - ETA: 5s - loss: 0.3044 - accuracy: 0.9153
 834/1500 [===============>..............] - ETA: 5s - loss: 0.3048 - accuracy: 0.9150
 841/1500 [===============>..............] - ETA: 5s - loss: 0.3043 - accuracy: 0.9150
 848/1500 [===============>..............] - ETA: 5s - loss: 0.3039 - accuracy: 0.9151
 854/1500 [================>.............] - ETA: 5s - loss: 0.3033 - accuracy: 0.9153
 861/1500 [================>.............] - ETA: 5s - loss: 0.3035 - accuracy: 0.9153
 868/1500 [================>.............] - ETA: 5s - loss: 0.3027 - accuracy: 0.9156
 875/1500 [================>.............] - ETA: 4s - loss: 0.3024 - accuracy: 0.9157
 882/1500 [================>.............] - ETA: 4s - loss: 0.3023 - accuracy: 0.9156
 889/1500 [================>.............] - ETA: 4s - loss: 0.3022 - accuracy: 0.9157
 896/1500 [================>.............] - ETA: 4s - loss: 0.3023 - accuracy: 0.9160
 903/1500 [=================>............] - ETA: 4s - loss: 0.3020 - accuracy: 0.9161
 910/1500 [=================>............] - ETA: 4s - loss: 0.3016 - accuracy: 0.9161
 917/1500 [=================>............] - ETA: 4s - loss: 0.3014 - accuracy: 0.9162
 924/1500 [=================>............] - ETA: 4s - loss: 0.3010 - accuracy: 0.9163
 932/1500 [=================>............] - ETA: 4s - loss: 0.3010 - accuracy: 0.9161
 939/1500 [=================>............] - ETA: 4s - loss: 0.3012 - accuracy: 0.9160
 946/1500 [=================>............] - ETA: 4s - loss: 0.3010 - accuracy: 0.9161
 953/1500 [==================>...........] - ETA: 4s - loss: 0.3008 - accuracy: 0.9163
 960/1500 [==================>...........] - ETA: 4s - loss: 0.3007 - accuracy: 0.9162
 967/1500 [==================>...........] - ETA: 4s - loss: 0.3002 - accuracy: 0.9162
 975/1500 [==================>...........] - ETA: 4s - loss: 0.3004 - accuracy: 0.9160
 982/1500 [==================>...........] - ETA: 4s - loss: 0.2999 - accuracy: 0.9161
 989/1500 [==================>...........] - ETA: 4s - loss: 0.2997 - accuracy: 0.9162
 996/1500 [==================>...........] - ETA: 3s - loss: 0.2999 - accuracy: 0.9162
1003/1500 [===================>..........] - ETA: 3s - loss: 0.3001 - accuracy: 0.9161
1010/1500 [===================>..........] - ETA: 3s - loss: 0.3007 - accuracy: 0.9160
1017/1500 [===================>..........] - ETA: 3s - loss: 0.3005 - accuracy: 0.9161
1023/1500 [===================>..........] - ETA: 3s - loss: 0.3011 - accuracy: 0.9159
1030/1500 [===================>..........] - ETA: 3s - loss: 0.3012 - accuracy: 0.9159
1037/1500 [===================>..........] - ETA: 3s - loss: 0.3015 - accuracy: 0.9158
1045/1500 [===================>..........] - ETA: 3s - loss: 0.3014 - accuracy: 0.9158
1052/1500 [====================>.........] - ETA: 3s - loss: 0.3014 - accuracy: 0.9158
1060/1500 [====================>.........] - ETA: 3s - loss: 0.3014 - accuracy: 0.9156
1068/1500 [====================>.........] - ETA: 3s - loss: 0.3017 - accuracy: 0.9155
1075/1500 [====================>.........] - ETA: 3s - loss: 0.3019 - accuracy: 0.9154
1082/1500 [====================>.........] - ETA: 3s - loss: 0.3016 - accuracy: 0.9155
1089/1500 [====================>.........] - ETA: 3s - loss: 0.3014 - accuracy: 0.9156
1095/1500 [====================>.........] - ETA: 3s - loss: 0.3014 - accuracy: 0.9156
1102/1500 [=====================>........] - ETA: 3s - loss: 0.3010 - accuracy: 0.9157
1109/1500 [=====================>........] - ETA: 3s - loss: 0.3017 - accuracy: 0.9156
1116/1500 [=====================>........] - ETA: 3s - loss: 0.3013 - accuracy: 0.9158
1123/1500 [=====================>........] - ETA: 2s - loss: 0.3009 - accuracy: 0.9159
1130/1500 [=====================>........] - ETA: 2s - loss: 0.3004 - accuracy: 0.9159
1137/1500 [=====================>........] - ETA: 2s - loss: 0.3002 - accuracy: 0.9161
1142/1500 [=====================>........] - ETA: 2s - loss: 0.3004 - accuracy: 0.9161
1149/1500 [=====================>........] - ETA: 2s - loss: 0.3003 - accuracy: 0.9161
1156/1500 [======================>.......] - ETA: 2s - loss: 0.3000 - accuracy: 0.9162
1163/1500 [======================>.......] - ETA: 2s - loss: 0.3000 - accuracy: 0.9162
1170/1500 [======================>.......] - ETA: 2s - loss: 0.3000 - accuracy: 0.9162
1177/1500 [======================>.......] - ETA: 2s - loss: 0.3003 - accuracy: 0.9161
1184/1500 [======================>.......] - ETA: 2s - loss: 0.3003 - accuracy: 0.9162
1191/1500 [======================>.......] - ETA: 2s - loss: 0.3002 - accuracy: 0.9162
1198/1500 [======================>.......] - ETA: 2s - loss: 0.2998 - accuracy: 0.9163
1205/1500 [=======================>......] - ETA: 2s - loss: 0.2999 - accuracy: 0.9164
1212/1500 [=======================>......] - ETA: 2s - loss: 0.2998 - accuracy: 0.9164
1219/1500 [=======================>......] - ETA: 2s - loss: 0.2996 - accuracy: 0.9163
1226/1500 [=======================>......] - ETA: 2s - loss: 0.2992 - accuracy: 0.9165
1233/1500 [=======================>......] - ETA: 2s - loss: 0.2992 - accuracy: 0.9165
1240/1500 [=======================>......] - ETA: 2s - loss: 0.2990 - accuracy: 0.9167
1247/1500 [=======================>......] - ETA: 1s - loss: 0.2989 - accuracy: 0.9167
1254/1500 [========================>.....] - ETA: 1s - loss: 0.2986 - accuracy: 0.9167
1261/1500 [========================>.....] - ETA: 1s - loss: 0.2988 - accuracy: 0.9167
1268/1500 [========================>.....] - ETA: 1s - loss: 0.2987 - accuracy: 0.9167
1275/1500 [========================>.....] - ETA: 1s - loss: 0.2989 - accuracy: 0.9166
1282/1500 [========================>.....] - ETA: 1s - loss: 0.2986 - accuracy: 0.9167
1289/1500 [========================>.....] - ETA: 1s - loss: 0.2985 - accuracy: 0.9166
1296/1500 [========================>.....] - ETA: 1s - loss: 0.2988 - accuracy: 0.9165
1303/1500 [=========================>....] - ETA: 1s - loss: 0.2990 - accuracy: 0.9164
1310/1500 [=========================>....] - ETA: 1s - loss: 0.2985 - accuracy: 0.9166
1317/1500 [=========================>....] - ETA: 1s - loss: 0.2979 - accuracy: 0.9167
1324/1500 [=========================>....] - ETA: 1s - loss: 0.2977 - accuracy: 0.9168
1331/1500 [=========================>....] - ETA: 1s - loss: 0.2977 - accuracy: 0.9169
1338/1500 [=========================>....] - ETA: 1s - loss: 0.2974 - accuracy: 0.9171
1345/1500 [=========================>....] - ETA: 1s - loss: 0.2970 - accuracy: 0.9171
1352/1500 [==========================>...] - ETA: 1s - loss: 0.2972 - accuracy: 0.9171
1359/1500 [==========================>...] - ETA: 1s - loss: 0.2971 - accuracy: 0.9172
1365/1500 [==========================>...] - ETA: 1s - loss: 0.2969 - accuracy: 0.9173
1372/1500 [==========================>...] - ETA: 1s - loss: 0.2969 - accuracy: 0.9172
1379/1500 [==========================>...] - ETA: 0s - loss: 0.2970 - accuracy: 0.9172
1386/1500 [==========================>...] - ETA: 0s - loss: 0.2971 - accuracy: 0.9170
1393/1500 [==========================>...] - ETA: 0s - loss: 0.2970 - accuracy: 0.9171
1400/1500 [===========================>..] - ETA: 0s - loss: 0.2965 - accuracy: 0.9171
1407/1500 [===========================>..] - ETA: 0s - loss: 0.2963 - accuracy: 0.9170
1414/1500 [===========================>..] - ETA: 0s - loss: 0.2959 - accuracy: 0.9171
1421/1500 [===========================>..] - ETA: 0s - loss: 0.2959 - accuracy: 0.9171
1428/1500 [===========================>..] - ETA: 0s - loss: 0.2955 - accuracy: 0.9172
1435/1500 [===========================>..] - ETA: 0s - loss: 0.2954 - accuracy: 0.9174
1442/1500 [===========================>..] - ETA: 0s - loss: 0.2956 - accuracy: 0.9173
1449/1500 [===========================>..] - ETA: 0s - loss: 0.2956 - accuracy: 0.9173
1456/1500 [============================>.] - ETA: 0s - loss: 0.2951 - accuracy: 0.9174
1463/1500 [============================>.] - ETA: 0s - loss: 0.2949 - accuracy: 0.9174
1470/1500 [============================>.] - ETA: 0s - loss: 0.2949 - accuracy: 0.9175
1478/1500 [============================>.] - ETA: 0s - loss: 0.2944 - accuracy: 0.9177
1485/1500 [============================>.] - ETA: 0s - loss: 0.2945 - accuracy: 0.9176
1492/1500 [============================>.] - ETA: 0s - loss: 0.2947 - accuracy: 0.9176
1498/1500 [============================>.] - ETA: 0s - loss: 0.2945 - accuracy: 0.9175
1500/1500 [==============================] - 12s 8ms/step - loss: 0.2945 - accuracy: 0.9175

1500/1500 [==============================] - 15s 10ms/step - loss: 0.2945 - accuracy: 0.9175 - val_loss: 0.2755 - val_accuracy: 0.9201
Epoch 4/10

   1/1500 [..............................] - ETA: 30s - loss: 0.1721 - accuracy: 0.9375
   9/1500 [..............................] - ETA: 10s - loss: 0.2813 - accuracy: 0.9201
  16/1500 [..............................] - ETA: 10s - loss: 0.2859 - accuracy: 0.9141
  24/1500 [..............................] - ETA: 10s - loss: 0.2976 - accuracy: 0.9193
  31/1500 [..............................] - ETA: 10s - loss: 0.2966 - accuracy: 0.9153
  38/1500 [..............................] - ETA: 10s - loss: 0.2905 - accuracy: 0.9194
  45/1500 [..............................] - ETA: 10s - loss: 0.3008 - accuracy: 0.9160
  52/1500 [>.............................] - ETA: 10s - loss: 0.3021 - accuracy: 0.9153
  58/1500 [>.............................] - ETA: 10s - loss: 0.3029 - accuracy: 0.9159
  65/1500 [>.............................] - ETA: 10s - loss: 0.3028 - accuracy: 0.9159
  72/1500 [>.............................] - ETA: 10s - loss: 0.2975 - accuracy: 0.9162
  80/1500 [>.............................] - ETA: 10s - loss: 0.2890 - accuracy: 0.9176
  88/1500 [>.............................] - ETA: 10s - loss: 0.2804 - accuracy: 0.9197
  96/1500 [>.............................] - ETA: 10s - loss: 0.2823 - accuracy: 0.9186
 104/1500 [=>............................] - ETA: 10s - loss: 0.2827 - accuracy: 0.9195
 112/1500 [=>............................] - ETA: 10s - loss: 0.2759 - accuracy: 0.9216
 120/1500 [=>............................] - ETA: 9s - loss: 0.2783 - accuracy: 0.9216 
 128/1500 [=>............................] - ETA: 9s - loss: 0.2733 - accuracy: 0.9233
 135/1500 [=>............................] - ETA: 9s - loss: 0.2741 - accuracy: 0.9236
 142/1500 [=>............................] - ETA: 9s - loss: 0.2735 - accuracy: 0.9241
 149/1500 [=>............................] - ETA: 9s - loss: 0.2773 - accuracy: 0.9222
 156/1500 [==>...........................] - ETA: 9s - loss: 0.2778 - accuracy: 0.9223
 163/1500 [==>...........................] - ETA: 9s - loss: 0.2780 - accuracy: 0.9224
 170/1500 [==>...........................] - ETA: 9s - loss: 0.2788 - accuracy: 0.9222
 176/1500 [==>...........................] - ETA: 9s - loss: 0.2786 - accuracy: 0.9229
 183/1500 [==>...........................] - ETA: 9s - loss: 0.2778 - accuracy: 0.9225
 190/1500 [==>...........................] - ETA: 9s - loss: 0.2776 - accuracy: 0.9222
 197/1500 [==>...........................] - ETA: 9s - loss: 0.2766 - accuracy: 0.9224
 204/1500 [===>..........................] - ETA: 9s - loss: 0.2754 - accuracy: 0.9226
 211/1500 [===>..........................] - ETA: 9s - loss: 0.2742 - accuracy: 0.9230
 218/1500 [===>..........................] - ETA: 9s - loss: 0.2743 - accuracy: 0.9227
 225/1500 [===>..........................] - ETA: 9s - loss: 0.2755 - accuracy: 0.9222
 231/1500 [===>..........................] - ETA: 9s - loss: 0.2765 - accuracy: 0.9223
 238/1500 [===>..........................] - ETA: 9s - loss: 0.2745 - accuracy: 0.9229
 245/1500 [===>..........................] - ETA: 9s - loss: 0.2755 - accuracy: 0.9227
 252/1500 [====>.........................] - ETA: 9s - loss: 0.2765 - accuracy: 0.9230
 259/1500 [====>.........................] - ETA: 9s - loss: 0.2761 - accuracy: 0.9233
 265/1500 [====>.........................] - ETA: 9s - loss: 0.2773 - accuracy: 0.9233
 272/1500 [====>.........................] - ETA: 9s - loss: 0.2784 - accuracy: 0.9231
 279/1500 [====>.........................] - ETA: 9s - loss: 0.2808 - accuracy: 0.9224
 286/1500 [====>.........................] - ETA: 9s - loss: 0.2825 - accuracy: 0.9215
 293/1500 [====>.........................] - ETA: 9s - loss: 0.2828 - accuracy: 0.9216
 300/1500 [=====>........................] - ETA: 9s - loss: 0.2826 - accuracy: 0.9215
 306/1500 [=====>........................] - ETA: 9s - loss: 0.2818 - accuracy: 0.9218
 313/1500 [=====>........................] - ETA: 9s - loss: 0.2804 - accuracy: 0.9224
 320/1500 [=====>........................] - ETA: 9s - loss: 0.2790 - accuracy: 0.9230
 327/1500 [=====>........................] - ETA: 8s - loss: 0.2785 - accuracy: 0.9227
 334/1500 [=====>........................] - ETA: 8s - loss: 0.2770 - accuracy: 0.9230
 340/1500 [=====>........................] - ETA: 8s - loss: 0.2760 - accuracy: 0.9232
 347/1500 [=====>........................] - ETA: 8s - loss: 0.2752 - accuracy: 0.9235
 353/1500 [======>.......................] - ETA: 8s - loss: 0.2753 - accuracy: 0.9234
 359/1500 [======>.......................] - ETA: 8s - loss: 0.2752 - accuracy: 0.9233
 365/1500 [======>.......................] - ETA: 8s - loss: 0.2754 - accuracy: 0.9235
 372/1500 [======>.......................] - ETA: 8s - loss: 0.2759 - accuracy: 0.9232
 379/1500 [======>.......................] - ETA: 8s - loss: 0.2759 - accuracy: 0.9236
 386/1500 [======>.......................] - ETA: 8s - loss: 0.2757 - accuracy: 0.9231
 391/1500 [======>.......................] - ETA: 8s - loss: 0.2746 - accuracy: 0.9235
 398/1500 [======>.......................] - ETA: 8s - loss: 0.2742 - accuracy: 0.9238
 404/1500 [=======>......................] - ETA: 8s - loss: 0.2733 - accuracy: 0.9240
 410/1500 [=======>......................] - ETA: 8s - loss: 0.2729 - accuracy: 0.9241
 417/1500 [=======>......................] - ETA: 8s - loss: 0.2720 - accuracy: 0.9241
 423/1500 [=======>......................] - ETA: 8s - loss: 0.2711 - accuracy: 0.9242
 430/1500 [=======>......................] - ETA: 8s - loss: 0.2702 - accuracy: 0.9246
 436/1500 [=======>......................] - ETA: 8s - loss: 0.2694 - accuracy: 0.9248
 443/1500 [=======>......................] - ETA: 8s - loss: 0.2684 - accuracy: 0.9251
 450/1500 [========>.....................] - ETA: 8s - loss: 0.2677 - accuracy: 0.9253
 456/1500 [========>.....................] - ETA: 8s - loss: 0.2673 - accuracy: 0.9254
 463/1500 [========>.....................] - ETA: 8s - loss: 0.2668 - accuracy: 0.9258
 470/1500 [========>.....................] - ETA: 8s - loss: 0.2665 - accuracy: 0.9256
 477/1500 [========>.....................] - ETA: 8s - loss: 0.2668 - accuracy: 0.9253
 484/1500 [========>.....................] - ETA: 8s - loss: 0.2666 - accuracy: 0.9255
 491/1500 [========>.....................] - ETA: 7s - loss: 0.2674 - accuracy: 0.9254
 498/1500 [========>.....................] - ETA: 7s - loss: 0.2686 - accuracy: 0.9250
 505/1500 [=========>....................] - ETA: 7s - loss: 0.2686 - accuracy: 0.9251
 511/1500 [=========>....................] - ETA: 7s - loss: 0.2695 - accuracy: 0.9248
 518/1500 [=========>....................] - ETA: 7s - loss: 0.2692 - accuracy: 0.9248
 525/1500 [=========>....................] - ETA: 7s - loss: 0.2687 - accuracy: 0.9251
 532/1500 [=========>....................] - ETA: 7s - loss: 0.2680 - accuracy: 0.9253
 539/1500 [=========>....................] - ETA: 7s - loss: 0.2690 - accuracy: 0.9248
 546/1500 [=========>....................] - ETA: 7s - loss: 0.2700 - accuracy: 0.9247
 553/1500 [==========>...................] - ETA: 7s - loss: 0.2697 - accuracy: 0.9246
 560/1500 [==========>...................] - ETA: 7s - loss: 0.2690 - accuracy: 0.9250
 566/1500 [==========>...................] - ETA: 7s - loss: 0.2696 - accuracy: 0.9247
 570/1500 [==========>...................] - ETA: 7s - loss: 0.2694 - accuracy: 0.9246
 577/1500 [==========>...................] - ETA: 7s - loss: 0.2692 - accuracy: 0.9247
 583/1500 [==========>...................] - ETA: 7s - loss: 0.2691 - accuracy: 0.9247
 590/1500 [==========>...................] - ETA: 7s - loss: 0.2685 - accuracy: 0.9249
 596/1500 [==========>...................] - ETA: 7s - loss: 0.2675 - accuracy: 0.9253
 602/1500 [===========>..................] - ETA: 7s - loss: 0.2675 - accuracy: 0.9250
 609/1500 [===========>..................] - ETA: 7s - loss: 0.2673 - accuracy: 0.9250
 615/1500 [===========>..................] - ETA: 7s - loss: 0.2667 - accuracy: 0.9252
 621/1500 [===========>..................] - ETA: 6s - loss: 0.2671 - accuracy: 0.9251
 628/1500 [===========>..................] - ETA: 6s - loss: 0.2658 - accuracy: 0.9256
 634/1500 [===========>..................] - ETA: 6s - loss: 0.2662 - accuracy: 0.9255
 640/1500 [===========>..................] - ETA: 6s - loss: 0.2656 - accuracy: 0.9257
 646/1500 [===========>..................] - ETA: 6s - loss: 0.2666 - accuracy: 0.9251
 653/1500 [============>.................] - ETA: 6s - loss: 0.2667 - accuracy: 0.9251
 659/1500 [============>.................] - ETA: 6s - loss: 0.2665 - accuracy: 0.9251
 666/1500 [============>.................] - ETA: 6s - loss: 0.2667 - accuracy: 0.9253
 673/1500 [============>.................] - ETA: 6s - loss: 0.2662 - accuracy: 0.9253
 680/1500 [============>.................] - ETA: 6s - loss: 0.2666 - accuracy: 0.9251
 687/1500 [============>.................] - ETA: 6s - loss: 0.2664 - accuracy: 0.9254
 694/1500 [============>.................] - ETA: 6s - loss: 0.2674 - accuracy: 0.9253
 701/1500 [=============>................] - ETA: 6s - loss: 0.2670 - accuracy: 0.9255
 708/1500 [=============>................] - ETA: 6s - loss: 0.2662 - accuracy: 0.9258
 715/1500 [=============>................] - ETA: 6s - loss: 0.2659 - accuracy: 0.9258
 722/1500 [=============>................] - ETA: 6s - loss: 0.2670 - accuracy: 0.9257
 728/1500 [=============>................] - ETA: 6s - loss: 0.2664 - accuracy: 0.9258
 735/1500 [=============>................] - ETA: 6s - loss: 0.2663 - accuracy: 0.9259
 742/1500 [=============>................] - ETA: 6s - loss: 0.2663 - accuracy: 0.9259
 749/1500 [=============>................] - ETA: 5s - loss: 0.2664 - accuracy: 0.9258
 756/1500 [==============>...............] - ETA: 5s - loss: 0.2664 - accuracy: 0.9258
 763/1500 [==============>...............] - ETA: 5s - loss: 0.2659 - accuracy: 0.9261
 770/1500 [==============>...............] - ETA: 5s - loss: 0.2652 - accuracy: 0.9262
 777/1500 [==============>...............] - ETA: 5s - loss: 0.2656 - accuracy: 0.9263
 784/1500 [==============>...............] - ETA: 5s - loss: 0.2659 - accuracy: 0.9262
 791/1500 [==============>...............] - ETA: 5s - loss: 0.2651 - accuracy: 0.9265
 798/1500 [==============>...............] - ETA: 5s - loss: 0.2649 - accuracy: 0.9264
 804/1500 [===============>..............] - ETA: 5s - loss: 0.2648 - accuracy: 0.9266
 811/1500 [===============>..............] - ETA: 5s - loss: 0.2646 - accuracy: 0.9267
 818/1500 [===============>..............] - ETA: 5s - loss: 0.2643 - accuracy: 0.9267
 825/1500 [===============>..............] - ETA: 5s - loss: 0.2642 - accuracy: 0.9267
 832/1500 [===============>..............] - ETA: 5s - loss: 0.2645 - accuracy: 0.9267
 839/1500 [===============>..............] - ETA: 5s - loss: 0.2650 - accuracy: 0.9265
 845/1500 [===============>..............] - ETA: 5s - loss: 0.2649 - accuracy: 0.9266
 852/1500 [================>.............] - ETA: 5s - loss: 0.2649 - accuracy: 0.9265
 859/1500 [================>.............] - ETA: 5s - loss: 0.2643 - accuracy: 0.9268
 866/1500 [================>.............] - ETA: 5s - loss: 0.2637 - accuracy: 0.9270
 873/1500 [================>.............] - ETA: 4s - loss: 0.2644 - accuracy: 0.9268
 880/1500 [================>.............] - ETA: 4s - loss: 0.2648 - accuracy: 0.9268
 887/1500 [================>.............] - ETA: 4s - loss: 0.2644 - accuracy: 0.9270
 894/1500 [================>.............] - ETA: 4s - loss: 0.2649 - accuracy: 0.9268
 901/1500 [=================>............] - ETA: 4s - loss: 0.2656 - accuracy: 0.9265
 907/1500 [=================>............] - ETA: 4s - loss: 0.2653 - accuracy: 0.9267
 914/1500 [=================>............] - ETA: 4s - loss: 0.2651 - accuracy: 0.9268
 921/1500 [=================>............] - ETA: 4s - loss: 0.2653 - accuracy: 0.9267
 928/1500 [=================>............] - ETA: 4s - loss: 0.2652 - accuracy: 0.9268
 935/1500 [=================>............] - ETA: 4s - loss: 0.2654 - accuracy: 0.9267
 942/1500 [=================>............] - ETA: 4s - loss: 0.2659 - accuracy: 0.9265
 949/1500 [=================>............] - ETA: 4s - loss: 0.2657 - accuracy: 0.9265
 956/1500 [==================>...........] - ETA: 4s - loss: 0.2653 - accuracy: 0.9268
 963/1500 [==================>...........] - ETA: 4s - loss: 0.2657 - accuracy: 0.9263
 970/1500 [==================>...........] - ETA: 4s - loss: 0.2652 - accuracy: 0.9264
 977/1500 [==================>...........] - ETA: 4s - loss: 0.2647 - accuracy: 0.9265
 983/1500 [==================>...........] - ETA: 4s - loss: 0.2649 - accuracy: 0.9264
 990/1500 [==================>...........] - ETA: 4s - loss: 0.2652 - accuracy: 0.9264
 997/1500 [==================>...........] - ETA: 3s - loss: 0.2653 - accuracy: 0.9264
1004/1500 [===================>..........] - ETA: 3s - loss: 0.2653 - accuracy: 0.9264
1011/1500 [===================>..........] - ETA: 3s - loss: 0.2655 - accuracy: 0.9264
1018/1500 [===================>..........] - ETA: 3s - loss: 0.2663 - accuracy: 0.9261
1025/1500 [===================>..........] - ETA: 3s - loss: 0.2662 - accuracy: 0.9261
1032/1500 [===================>..........] - ETA: 3s - loss: 0.2659 - accuracy: 0.9261
1038/1500 [===================>..........] - ETA: 3s - loss: 0.2657 - accuracy: 0.9262
1045/1500 [===================>..........] - ETA: 3s - loss: 0.2659 - accuracy: 0.9262
1052/1500 [====================>.........] - ETA: 3s - loss: 0.2654 - accuracy: 0.9264
1059/1500 [====================>.........] - ETA: 3s - loss: 0.2653 - accuracy: 0.9264
1066/1500 [====================>.........] - ETA: 3s - loss: 0.2653 - accuracy: 0.9263
1073/1500 [====================>.........] - ETA: 3s - loss: 0.2654 - accuracy: 0.9262
1080/1500 [====================>.........] - ETA: 3s - loss: 0.2657 - accuracy: 0.9262
1087/1500 [====================>.........] - ETA: 3s - loss: 0.2655 - accuracy: 0.9263
1093/1500 [====================>.........] - ETA: 3s - loss: 0.2654 - accuracy: 0.9263
1099/1500 [====================>.........] - ETA: 3s - loss: 0.2657 - accuracy: 0.9261
1106/1500 [=====================>........] - ETA: 3s - loss: 0.2654 - accuracy: 0.9261
1114/1500 [=====================>........] - ETA: 3s - loss: 0.2662 - accuracy: 0.9259
1121/1500 [=====================>........] - ETA: 3s - loss: 0.2663 - accuracy: 0.9257
1128/1500 [=====================>........] - ETA: 2s - loss: 0.2662 - accuracy: 0.9257
1135/1500 [=====================>........] - ETA: 2s - loss: 0.2661 - accuracy: 0.9257
1141/1500 [=====================>........] - ETA: 2s - loss: 0.2659 - accuracy: 0.9258
1148/1500 [=====================>........] - ETA: 2s - loss: 0.2658 - accuracy: 0.9259
1155/1500 [======================>.......] - ETA: 2s - loss: 0.2655 - accuracy: 0.9260
1162/1500 [======================>.......] - ETA: 2s - loss: 0.2658 - accuracy: 0.9257
1169/1500 [======================>.......] - ETA: 2s - loss: 0.2658 - accuracy: 0.9258
1176/1500 [======================>.......] - ETA: 2s - loss: 0.2662 - accuracy: 0.9256
1183/1500 [======================>.......] - ETA: 2s - loss: 0.2662 - accuracy: 0.9257
1189/1500 [======================>.......] - ETA: 2s - loss: 0.2660 - accuracy: 0.9258
1195/1500 [======================>.......] - ETA: 2s - loss: 0.2661 - accuracy: 0.9259
1202/1500 [=======================>......] - ETA: 2s - loss: 0.2663 - accuracy: 0.9258
1209/1500 [=======================>......] - ETA: 2s - loss: 0.2667 - accuracy: 0.9258
1216/1500 [=======================>......] - ETA: 2s - loss: 0.2664 - accuracy: 0.9259
1223/1500 [=======================>......] - ETA: 2s - loss: 0.2661 - accuracy: 0.9259
1230/1500 [=======================>......] - ETA: 2s - loss: 0.2661 - accuracy: 0.9260
1237/1500 [=======================>......] - ETA: 2s - loss: 0.2661 - accuracy: 0.9260
1244/1500 [=======================>......] - ETA: 2s - loss: 0.2656 - accuracy: 0.9261
1251/1500 [========================>.....] - ETA: 1s - loss: 0.2659 - accuracy: 0.9261
1258/1500 [========================>.....] - ETA: 1s - loss: 0.2655 - accuracy: 0.9262
1265/1500 [========================>.....] - ETA: 1s - loss: 0.2651 - accuracy: 0.9262
1272/1500 [========================>.....] - ETA: 1s - loss: 0.2648 - accuracy: 0.9262
1278/1500 [========================>.....] - ETA: 1s - loss: 0.2647 - accuracy: 0.9263
1285/1500 [========================>.....] - ETA: 1s - loss: 0.2642 - accuracy: 0.9264
1291/1500 [========================>.....] - ETA: 1s - loss: 0.2640 - accuracy: 0.9264
1298/1500 [========================>.....] - ETA: 1s - loss: 0.2637 - accuracy: 0.9265
1305/1500 [=========================>....] - ETA: 1s - loss: 0.2633 - accuracy: 0.9267
1312/1500 [=========================>....] - ETA: 1s - loss: 0.2632 - accuracy: 0.9268
1318/1500 [=========================>....] - ETA: 1s - loss: 0.2632 - accuracy: 0.9268
1325/1500 [=========================>....] - ETA: 1s - loss: 0.2634 - accuracy: 0.9266
1331/1500 [=========================>....] - ETA: 1s - loss: 0.2642 - accuracy: 0.9264
1338/1500 [=========================>....] - ETA: 1s - loss: 0.2640 - accuracy: 0.9265
1345/1500 [=========================>....] - ETA: 1s - loss: 0.2642 - accuracy: 0.9265
1352/1500 [==========================>...] - ETA: 1s - loss: 0.2639 - accuracy: 0.9266
1359/1500 [==========================>...] - ETA: 1s - loss: 0.2641 - accuracy: 0.9264
1366/1500 [==========================>...] - ETA: 1s - loss: 0.2642 - accuracy: 0.9264
1373/1500 [==========================>...] - ETA: 1s - loss: 0.2640 - accuracy: 0.9265
1380/1500 [==========================>...] - ETA: 0s - loss: 0.2639 - accuracy: 0.9265
1387/1500 [==========================>...] - ETA: 0s - loss: 0.2635 - accuracy: 0.9266
1394/1500 [==========================>...] - ETA: 0s - loss: 0.2641 - accuracy: 0.9265
1401/1500 [===========================>..] - ETA: 0s - loss: 0.2643 - accuracy: 0.9265
1408/1500 [===========================>..] - ETA: 0s - loss: 0.2642 - accuracy: 0.9265
1415/1500 [===========================>..] - ETA: 0s - loss: 0.2642 - accuracy: 0.9266
1422/1500 [===========================>..] - ETA: 0s - loss: 0.2643 - accuracy: 0.9265
1429/1500 [===========================>..] - ETA: 0s - loss: 0.2639 - accuracy: 0.9266
1436/1500 [===========================>..] - ETA: 0s - loss: 0.2640 - accuracy: 0.9266
1443/1500 [===========================>..] - ETA: 0s - loss: 0.2636 - accuracy: 0.9268
1450/1500 [============================>.] - ETA: 0s - loss: 0.2634 - accuracy: 0.9269
1457/1500 [============================>.] - ETA: 0s - loss: 0.2637 - accuracy: 0.9269
1464/1500 [============================>.] - ETA: 0s - loss: 0.2638 - accuracy: 0.9270
1471/1500 [============================>.] - ETA: 0s - loss: 0.2639 - accuracy: 0.9269
1478/1500 [============================>.] - ETA: 0s - loss: 0.2639 - accuracy: 0.9270
1485/1500 [============================>.] - ETA: 0s - loss: 0.2640 - accuracy: 0.9270
1492/1500 [============================>.] - ETA: 0s - loss: 0.2641 - accuracy: 0.9269
1499/1500 [============================>.] - ETA: 0s - loss: 0.2638 - accuracy: 0.9271
1500/1500 [==============================] - 12s 8ms/step - loss: 0.2637 - accuracy: 0.9271

1500/1500 [==============================] - 15s 10ms/step - loss: 0.2637 - accuracy: 0.9271 - val_loss: 0.2497 - val_accuracy: 0.9286
Epoch 5/10

   1/1500 [..............................] - ETA: 26s - loss: 0.2630 - accuracy: 0.9375
   8/1500 [..............................] - ETA: 10s - loss: 0.2623 - accuracy: 0.9414
  15/1500 [..............................] - ETA: 10s - loss: 0.2549 - accuracy: 0.9438
  22/1500 [..............................] - ETA: 10s - loss: 0.2481 - accuracy: 0.9361
  30/1500 [..............................] - ETA: 10s - loss: 0.2534 - accuracy: 0.9281
  36/1500 [..............................] - ETA: 10s - loss: 0.2401 - accuracy: 0.9314
  43/1500 [..............................] - ETA: 11s - loss: 0.2382 - accuracy: 0.9317
  50/1500 [>.............................] - ETA: 11s - loss: 0.2397 - accuracy: 0.9319
  57/1500 [>.............................] - ETA: 11s - loss: 0.2350 - accuracy: 0.9342
  64/1500 [>.............................] - ETA: 11s - loss: 0.2309 - accuracy: 0.9355
  71/1500 [>.............................] - ETA: 10s - loss: 0.2354 - accuracy: 0.9335
  79/1500 [>.............................] - ETA: 10s - loss: 0.2420 - accuracy: 0.9312
  86/1500 [>.............................] - ETA: 10s - loss: 0.2421 - accuracy: 0.9310
  93/1500 [>.............................] - ETA: 10s - loss: 0.2413 - accuracy: 0.9311
 100/1500 [=>............................] - ETA: 10s - loss: 0.2455 - accuracy: 0.9294
 107/1500 [=>............................] - ETA: 10s - loss: 0.2403 - accuracy: 0.9311
 114/1500 [=>............................] - ETA: 10s - loss: 0.2398 - accuracy: 0.9301
 121/1500 [=>............................] - ETA: 10s - loss: 0.2431 - accuracy: 0.9295
 128/1500 [=>............................] - ETA: 10s - loss: 0.2465 - accuracy: 0.9272
 135/1500 [=>............................] - ETA: 10s - loss: 0.2430 - accuracy: 0.9282
 141/1500 [=>............................] - ETA: 10s - loss: 0.2393 - accuracy: 0.9302
 147/1500 [=>............................] - ETA: 10s - loss: 0.2411 - accuracy: 0.9298
 154/1500 [==>...........................] - ETA: 10s - loss: 0.2470 - accuracy: 0.9280
 161/1500 [==>...........................] - ETA: 10s - loss: 0.2466 - accuracy: 0.9284
 168/1500 [==>...........................] - ETA: 10s - loss: 0.2455 - accuracy: 0.9289
 175/1500 [==>...........................] - ETA: 10s - loss: 0.2459 - accuracy: 0.9289
 182/1500 [==>...........................] - ETA: 10s - loss: 0.2457 - accuracy: 0.9289
 189/1500 [==>...........................] - ETA: 10s - loss: 0.2477 - accuracy: 0.9284
 196/1500 [==>...........................] - ETA: 10s - loss: 0.2462 - accuracy: 0.9290
 203/1500 [===>..........................] - ETA: 10s - loss: 0.2467 - accuracy: 0.9289
 209/1500 [===>..........................] - ETA: 10s - loss: 0.2478 - accuracy: 0.9287
 213/1500 [===>..........................] - ETA: 10s - loss: 0.2501 - accuracy: 0.9281
 220/1500 [===>..........................] - ETA: 10s - loss: 0.2491 - accuracy: 0.9290
 227/1500 [===>..........................] - ETA: 9s - loss: 0.2493 - accuracy: 0.9294 
 234/1500 [===>..........................] - ETA: 9s - loss: 0.2497 - accuracy: 0.9292
 241/1500 [===>..........................] - ETA: 9s - loss: 0.2489 - accuracy: 0.9291
 248/1500 [===>..........................] - ETA: 9s - loss: 0.2480 - accuracy: 0.9293
 255/1500 [====>.........................] - ETA: 9s - loss: 0.2492 - accuracy: 0.9288
 262/1500 [====>.........................] - ETA: 9s - loss: 0.2486 - accuracy: 0.9288
 268/1500 [====>.........................] - ETA: 9s - loss: 0.2472 - accuracy: 0.9293
 275/1500 [====>.........................] - ETA: 9s - loss: 0.2460 - accuracy: 0.9298
 282/1500 [====>.........................] - ETA: 9s - loss: 0.2469 - accuracy: 0.9294
 289/1500 [====>.........................] - ETA: 9s - loss: 0.2470 - accuracy: 0.9290
 296/1500 [====>.........................] - ETA: 9s - loss: 0.2467 - accuracy: 0.9292
 303/1500 [=====>........................] - ETA: 9s - loss: 0.2452 - accuracy: 0.9298
 310/1500 [=====>........................] - ETA: 9s - loss: 0.2466 - accuracy: 0.9298
 317/1500 [=====>........................] - ETA: 9s - loss: 0.2463 - accuracy: 0.9301
 324/1500 [=====>........................] - ETA: 9s - loss: 0.2470 - accuracy: 0.9303
 331/1500 [=====>........................] - ETA: 9s - loss: 0.2478 - accuracy: 0.9299
 338/1500 [=====>........................] - ETA: 9s - loss: 0.2458 - accuracy: 0.9307
 344/1500 [=====>........................] - ETA: 9s - loss: 0.2464 - accuracy: 0.9305
 351/1500 [======>.......................] - ETA: 9s - loss: 0.2460 - accuracy: 0.9307
 358/1500 [======>.......................] - ETA: 9s - loss: 0.2465 - accuracy: 0.9306
 365/1500 [======>.......................] - ETA: 8s - loss: 0.2467 - accuracy: 0.9303
 371/1500 [======>.......................] - ETA: 8s - loss: 0.2462 - accuracy: 0.9307
 378/1500 [======>.......................] - ETA: 8s - loss: 0.2462 - accuracy: 0.9306
 385/1500 [======>.......................] - ETA: 8s - loss: 0.2468 - accuracy: 0.9304
 392/1500 [======>.......................] - ETA: 8s - loss: 0.2467 - accuracy: 0.9304
 399/1500 [======>.......................] - ETA: 8s - loss: 0.2467 - accuracy: 0.9304
 405/1500 [=======>......................] - ETA: 8s - loss: 0.2474 - accuracy: 0.9302
 412/1500 [=======>......................] - ETA: 8s - loss: 0.2475 - accuracy: 0.9301
 419/1500 [=======>......................] - ETA: 8s - loss: 0.2481 - accuracy: 0.9300
 426/1500 [=======>......................] - ETA: 8s - loss: 0.2472 - accuracy: 0.9302
 433/1500 [=======>......................] - ETA: 8s - loss: 0.2464 - accuracy: 0.9304
 440/1500 [=======>......................] - ETA: 8s - loss: 0.2470 - accuracy: 0.9302
 447/1500 [=======>......................] - ETA: 8s - loss: 0.2473 - accuracy: 0.9301
 453/1500 [========>.....................] - ETA: 8s - loss: 0.2476 - accuracy: 0.9299
 460/1500 [========>.....................] - ETA: 8s - loss: 0.2473 - accuracy: 0.9299
 466/1500 [========>.....................] - ETA: 8s - loss: 0.2472 - accuracy: 0.9300
 473/1500 [========>.....................] - ETA: 8s - loss: 0.2476 - accuracy: 0.9299
 480/1500 [========>.....................] - ETA: 8s - loss: 0.2475 - accuracy: 0.9299
 487/1500 [========>.....................] - ETA: 8s - loss: 0.2489 - accuracy: 0.9295
 494/1500 [========>.....................] - ETA: 7s - loss: 0.2491 - accuracy: 0.9296
 500/1500 [=========>....................] - ETA: 7s - loss: 0.2489 - accuracy: 0.9294
 507/1500 [=========>....................] - ETA: 7s - loss: 0.2485 - accuracy: 0.9293
 514/1500 [=========>....................] - ETA: 7s - loss: 0.2495 - accuracy: 0.9288
 521/1500 [=========>....................] - ETA: 7s - loss: 0.2486 - accuracy: 0.9292
 528/1500 [=========>....................] - ETA: 7s - loss: 0.2483 - accuracy: 0.9294
 535/1500 [=========>....................] - ETA: 7s - loss: 0.2482 - accuracy: 0.9293
 541/1500 [=========>....................] - ETA: 7s - loss: 0.2476 - accuracy: 0.9294
 548/1500 [=========>....................] - ETA: 7s - loss: 0.2484 - accuracy: 0.9292
 554/1500 [==========>...................] - ETA: 7s - loss: 0.2486 - accuracy: 0.9290
 556/1500 [==========>...................] - ETA: 7s - loss: 0.2485 - accuracy: 0.9290
 563/1500 [==========>...................] - ETA: 7s - loss: 0.2485 - accuracy: 0.9288
 570/1500 [==========>...................] - ETA: 7s - loss: 0.2486 - accuracy: 0.9288
 577/1500 [==========>...................] - ETA: 7s - loss: 0.2484 - accuracy: 0.9291
 582/1500 [==========>...................] - ETA: 7s - loss: 0.2480 - accuracy: 0.9292
 585/1500 [==========>...................] - ETA: 7s - loss: 0.2479 - accuracy: 0.9292
 592/1500 [==========>...................] - ETA: 7s - loss: 0.2484 - accuracy: 0.9291
 599/1500 [==========>...................] - ETA: 7s - loss: 0.2480 - accuracy: 0.9292
 606/1500 [===========>..................] - ETA: 7s - loss: 0.2479 - accuracy: 0.9295
 612/1500 [===========>..................] - ETA: 7s - loss: 0.2482 - accuracy: 0.9294
 619/1500 [===========>..................] - ETA: 7s - loss: 0.2486 - accuracy: 0.9293
 625/1500 [===========>..................] - ETA: 7s - loss: 0.2481 - accuracy: 0.9295
 632/1500 [===========>..................] - ETA: 7s - loss: 0.2484 - accuracy: 0.9296
 638/1500 [===========>..................] - ETA: 7s - loss: 0.2485 - accuracy: 0.9296
 644/1500 [===========>..................] - ETA: 6s - loss: 0.2479 - accuracy: 0.9298
 650/1500 [============>.................] - ETA: 6s - loss: 0.2474 - accuracy: 0.9300
 657/1500 [============>.................] - ETA: 6s - loss: 0.2472 - accuracy: 0.9302
 664/1500 [============>.................] - ETA: 6s - loss: 0.2468 - accuracy: 0.9303
 671/1500 [============>.................] - ETA: 6s - loss: 0.2468 - accuracy: 0.9302
 678/1500 [============>.................] - ETA: 6s - loss: 0.2466 - accuracy: 0.9304
 685/1500 [============>.................] - ETA: 6s - loss: 0.2464 - accuracy: 0.9305
 692/1500 [============>.................] - ETA: 6s - loss: 0.2466 - accuracy: 0.9301
 698/1500 [============>.................] - ETA: 6s - loss: 0.2465 - accuracy: 0.9302
 704/1500 [=============>................] - ETA: 6s - loss: 0.2455 - accuracy: 0.9305
 711/1500 [=============>................] - ETA: 6s - loss: 0.2458 - accuracy: 0.9306
 718/1500 [=============>................] - ETA: 6s - loss: 0.2450 - accuracy: 0.9309
 725/1500 [=============>................] - ETA: 6s - loss: 0.2446 - accuracy: 0.9312
 732/1500 [=============>................] - ETA: 6s - loss: 0.2446 - accuracy: 0.9312
 739/1500 [=============>................] - ETA: 6s - loss: 0.2451 - accuracy: 0.9311
 746/1500 [=============>................] - ETA: 6s - loss: 0.2454 - accuracy: 0.9310
 753/1500 [==============>...............] - ETA: 6s - loss: 0.2452 - accuracy: 0.9311
 760/1500 [==============>...............] - ETA: 6s - loss: 0.2451 - accuracy: 0.9312
 766/1500 [==============>...............] - ETA: 5s - loss: 0.2448 - accuracy: 0.9313
 771/1500 [==============>...............] - ETA: 5s - loss: 0.2446 - accuracy: 0.9315
 777/1500 [==============>...............] - ETA: 5s - loss: 0.2442 - accuracy: 0.9316
 784/1500 [==============>...............] - ETA: 5s - loss: 0.2440 - accuracy: 0.9318
 791/1500 [==============>...............] - ETA: 5s - loss: 0.2438 - accuracy: 0.9317
 798/1500 [==============>...............] - ETA: 5s - loss: 0.2433 - accuracy: 0.9320
 805/1500 [===============>..............] - ETA: 5s - loss: 0.2434 - accuracy: 0.9320
 812/1500 [===============>..............] - ETA: 5s - loss: 0.2435 - accuracy: 0.9322
 819/1500 [===============>..............] - ETA: 5s - loss: 0.2438 - accuracy: 0.9321
 826/1500 [===============>..............] - ETA: 5s - loss: 0.2439 - accuracy: 0.9320
 833/1500 [===============>..............] - ETA: 5s - loss: 0.2444 - accuracy: 0.9318
 840/1500 [===============>..............] - ETA: 5s - loss: 0.2446 - accuracy: 0.9320
 847/1500 [===============>..............] - ETA: 5s - loss: 0.2446 - accuracy: 0.9319
 854/1500 [================>.............] - ETA: 5s - loss: 0.2442 - accuracy: 0.9320
 861/1500 [================>.............] - ETA: 5s - loss: 0.2444 - accuracy: 0.9320
 868/1500 [================>.............] - ETA: 5s - loss: 0.2441 - accuracy: 0.9320
 876/1500 [================>.............] - ETA: 5s - loss: 0.2440 - accuracy: 0.9320
 883/1500 [================>.............] - ETA: 5s - loss: 0.2443 - accuracy: 0.9318
 890/1500 [================>.............] - ETA: 4s - loss: 0.2439 - accuracy: 0.9320
 897/1500 [================>.............] - ETA: 4s - loss: 0.2439 - accuracy: 0.9321
 903/1500 [=================>............] - ETA: 4s - loss: 0.2436 - accuracy: 0.9322
 910/1500 [=================>............] - ETA: 4s - loss: 0.2435 - accuracy: 0.9321
 916/1500 [=================>............] - ETA: 4s - loss: 0.2435 - accuracy: 0.9322
 922/1500 [=================>............] - ETA: 4s - loss: 0.2434 - accuracy: 0.9323
 928/1500 [=================>............] - ETA: 4s - loss: 0.2432 - accuracy: 0.9324
 935/1500 [=================>............] - ETA: 4s - loss: 0.2429 - accuracy: 0.9325
 941/1500 [=================>............] - ETA: 4s - loss: 0.2426 - accuracy: 0.9327
 948/1500 [=================>............] - ETA: 4s - loss: 0.2427 - accuracy: 0.9327
 955/1500 [==================>...........] - ETA: 4s - loss: 0.2424 - accuracy: 0.9329
 961/1500 [==================>...........] - ETA: 4s - loss: 0.2428 - accuracy: 0.9328
 967/1500 [==================>...........] - ETA: 4s - loss: 0.2428 - accuracy: 0.9327
 974/1500 [==================>...........] - ETA: 4s - loss: 0.2430 - accuracy: 0.9327
 980/1500 [==================>...........] - ETA: 4s - loss: 0.2426 - accuracy: 0.9327
 986/1500 [==================>...........] - ETA: 4s - loss: 0.2423 - accuracy: 0.9328
 992/1500 [==================>...........] - ETA: 4s - loss: 0.2425 - accuracy: 0.9328
 999/1500 [==================>...........] - ETA: 4s - loss: 0.2421 - accuracy: 0.9329
1006/1500 [===================>..........] - ETA: 4s - loss: 0.2419 - accuracy: 0.9329
1012/1500 [===================>..........] - ETA: 3s - loss: 0.2422 - accuracy: 0.9328
1018/1500 [===================>..........] - ETA: 3s - loss: 0.2426 - accuracy: 0.9328
1025/1500 [===================>..........] - ETA: 3s - loss: 0.2433 - accuracy: 0.9326
1031/1500 [===================>..........] - ETA: 3s - loss: 0.2432 - accuracy: 0.9327
1038/1500 [===================>..........] - ETA: 3s - loss: 0.2430 - accuracy: 0.9326
1044/1500 [===================>..........] - ETA: 3s - loss: 0.2436 - accuracy: 0.9324
1051/1500 [====================>.........] - ETA: 3s - loss: 0.2433 - accuracy: 0.9325
1057/1500 [====================>.........] - ETA: 3s - loss: 0.2432 - accuracy: 0.9325
1064/1500 [====================>.........] - ETA: 3s - loss: 0.2434 - accuracy: 0.9324
1071/1500 [====================>.........] - ETA: 3s - loss: 0.2434 - accuracy: 0.9323
1077/1500 [====================>.........] - ETA: 3s - loss: 0.2433 - accuracy: 0.9323
1084/1500 [====================>.........] - ETA: 3s - loss: 0.2429 - accuracy: 0.9325
1091/1500 [====================>.........] - ETA: 3s - loss: 0.2429 - accuracy: 0.9325
1098/1500 [====================>.........] - ETA: 3s - loss: 0.2425 - accuracy: 0.9326
1105/1500 [=====================>........] - ETA: 3s - loss: 0.2431 - accuracy: 0.9324
1111/1500 [=====================>........] - ETA: 3s - loss: 0.2428 - accuracy: 0.9325
1114/1500 [=====================>........] - ETA: 3s - loss: 0.2429 - accuracy: 0.9325
1120/1500 [=====================>........] - ETA: 3s - loss: 0.2432 - accuracy: 0.9324
1126/1500 [=====================>........] - ETA: 3s - loss: 0.2437 - accuracy: 0.9323
1132/1500 [=====================>........] - ETA: 3s - loss: 0.2435 - accuracy: 0.9323
1139/1500 [=====================>........] - ETA: 2s - loss: 0.2432 - accuracy: 0.9323
1145/1500 [=====================>........] - ETA: 2s - loss: 0.2432 - accuracy: 0.9323
1152/1500 [======================>.......] - ETA: 2s - loss: 0.2430 - accuracy: 0.9324
1159/1500 [======================>.......] - ETA: 2s - loss: 0.2428 - accuracy: 0.9325
1166/1500 [======================>.......] - ETA: 2s - loss: 0.2426 - accuracy: 0.9326
1173/1500 [======================>.......] - ETA: 2s - loss: 0.2422 - accuracy: 0.9328
1180/1500 [======================>.......] - ETA: 2s - loss: 0.2425 - accuracy: 0.9328
1186/1500 [======================>.......] - ETA: 2s - loss: 0.2426 - accuracy: 0.9327
1193/1500 [======================>.......] - ETA: 2s - loss: 0.2426 - accuracy: 0.9326
1200/1500 [=======================>......] - ETA: 2s - loss: 0.2425 - accuracy: 0.9327
1207/1500 [=======================>......] - ETA: 2s - loss: 0.2423 - accuracy: 0.9328
1214/1500 [=======================>......] - ETA: 2s - loss: 0.2421 - accuracy: 0.9330
1221/1500 [=======================>......] - ETA: 2s - loss: 0.2419 - accuracy: 0.9331
1227/1500 [=======================>......] - ETA: 2s - loss: 0.2418 - accuracy: 0.9330
1233/1500 [=======================>......] - ETA: 2s - loss: 0.2419 - accuracy: 0.9330
1240/1500 [=======================>......] - ETA: 2s - loss: 0.2414 - accuracy: 0.9332
1247/1500 [=======================>......] - ETA: 2s - loss: 0.2414 - accuracy: 0.9332
1254/1500 [========================>.....] - ETA: 2s - loss: 0.2413 - accuracy: 0.9332
1261/1500 [========================>.....] - ETA: 1s - loss: 0.2413 - accuracy: 0.9333
1267/1500 [========================>.....] - ETA: 1s - loss: 0.2412 - accuracy: 0.9333
1274/1500 [========================>.....] - ETA: 1s - loss: 0.2414 - accuracy: 0.9332
1281/1500 [========================>.....] - ETA: 1s - loss: 0.2414 - accuracy: 0.9332
1288/1500 [========================>.....] - ETA: 1s - loss: 0.2412 - accuracy: 0.9332
1295/1500 [========================>.....] - ETA: 1s - loss: 0.2411 - accuracy: 0.9333
1302/1500 [=========================>....] - ETA: 1s - loss: 0.2409 - accuracy: 0.9333
1308/1500 [=========================>....] - ETA: 1s - loss: 0.2408 - accuracy: 0.9333
1315/1500 [=========================>....] - ETA: 1s - loss: 0.2408 - accuracy: 0.9333
1322/1500 [=========================>....] - ETA: 1s - loss: 0.2408 - accuracy: 0.9333
1328/1500 [=========================>....] - ETA: 1s - loss: 0.2407 - accuracy: 0.9333
1335/1500 [=========================>....] - ETA: 1s - loss: 0.2406 - accuracy: 0.9334
1342/1500 [=========================>....] - ETA: 1s - loss: 0.2405 - accuracy: 0.9334
1349/1500 [=========================>....] - ETA: 1s - loss: 0.2408 - accuracy: 0.9334
1356/1500 [==========================>...] - ETA: 1s - loss: 0.2408 - accuracy: 0.9334
1363/1500 [==========================>...] - ETA: 1s - loss: 0.2407 - accuracy: 0.9334
1370/1500 [==========================>...] - ETA: 1s - loss: 0.2406 - accuracy: 0.9334
1377/1500 [==========================>...] - ETA: 1s - loss: 0.2404 - accuracy: 0.9334
1384/1500 [==========================>...] - ETA: 0s - loss: 0.2409 - accuracy: 0.9332
1391/1500 [==========================>...] - ETA: 0s - loss: 0.2410 - accuracy: 0.9332
1398/1500 [==========================>...] - ETA: 0s - loss: 0.2407 - accuracy: 0.9333
1405/1500 [===========================>..] - ETA: 0s - loss: 0.2406 - accuracy: 0.9333
1412/1500 [===========================>..] - ETA: 0s - loss: 0.2403 - accuracy: 0.9335
1419/1500 [===========================>..] - ETA: 0s - loss: 0.2401 - accuracy: 0.9335
1426/1500 [===========================>..] - ETA: 0s - loss: 0.2402 - accuracy: 0.9335
1433/1500 [===========================>..] - ETA: 0s - loss: 0.2401 - accuracy: 0.9335
1440/1500 [===========================>..] - ETA: 0s - loss: 0.2401 - accuracy: 0.9335
1447/1500 [===========================>..] - ETA: 0s - loss: 0.2402 - accuracy: 0.9335
1454/1500 [============================>.] - ETA: 0s - loss: 0.2402 - accuracy: 0.9335
1461/1500 [============================>.] - ETA: 0s - loss: 0.2402 - accuracy: 0.9334
1467/1500 [============================>.] - ETA: 0s - loss: 0.2400 - accuracy: 0.9334
1473/1500 [============================>.] - ETA: 0s - loss: 0.2403 - accuracy: 0.9333
1480/1500 [============================>.] - ETA: 0s - loss: 0.2401 - accuracy: 0.9333
1487/1500 [============================>.] - ETA: 0s - loss: 0.2402 - accuracy: 0.9333
1493/1500 [============================>.] - ETA: 0s - loss: 0.2402 - accuracy: 0.9333
1499/1500 [============================>.] - ETA: 0s - loss: 0.2400 - accuracy: 0.9334
1500/1500 [==============================] - 12s 8ms/step - loss: 0.2400 - accuracy: 0.9334

1500/1500 [==============================] - 15s 10ms/step - loss: 0.2400 - accuracy: 0.9334 - val_loss: 0.2308 - val_accuracy: 0.9344
Epoch 6/10

   1/1500 [..............................] - ETA: 45s - loss: 0.1673 - accuracy: 0.9375
   8/1500 [..............................] - ETA: 10s - loss: 0.1984 - accuracy: 0.9531
  15/1500 [..............................] - ETA: 11s - loss: 0.2413 - accuracy: 0.9438
  23/1500 [..............................] - ETA: 10s - loss: 0.2231 - accuracy: 0.9484
  30/1500 [..............................] - ETA: 10s - loss: 0.2190 - accuracy: 0.9479
  37/1500 [..............................] - ETA: 10s - loss: 0.2155 - accuracy: 0.9459
  44/1500 [..............................] - ETA: 10s - loss: 0.2172 - accuracy: 0.9460
  51/1500 [>.............................] - ETA: 10s - loss: 0.2164 - accuracy: 0.9442
  58/1500 [>.............................] - ETA: 10s - loss: 0.2239 - accuracy: 0.9434
  65/1500 [>.............................] - ETA: 10s - loss: 0.2184 - accuracy: 0.9442
  72/1500 [>.............................] - ETA: 10s - loss: 0.2175 - accuracy: 0.9444
  79/1500 [>.............................] - ETA: 10s - loss: 0.2180 - accuracy: 0.9422
  86/1500 [>.............................] - ETA: 10s - loss: 0.2191 - accuracy: 0.9422
  93/1500 [>.............................] - ETA: 10s - loss: 0.2233 - accuracy: 0.9399
 100/1500 [=>............................] - ETA: 10s - loss: 0.2255 - accuracy: 0.9387
 107/1500 [=>............................] - ETA: 10s - loss: 0.2248 - accuracy: 0.9393
 114/1500 [=>............................] - ETA: 10s - loss: 0.2265 - accuracy: 0.9380
 120/1500 [=>............................] - ETA: 10s - loss: 0.2239 - accuracy: 0.9388
 126/1500 [=>............................] - ETA: 10s - loss: 0.2236 - accuracy: 0.9385
 133/1500 [=>............................] - ETA: 10s - loss: 0.2235 - accuracy: 0.9382
 138/1500 [=>............................] - ETA: 10s - loss: 0.2222 - accuracy: 0.9386
 145/1500 [=>............................] - ETA: 10s - loss: 0.2184 - accuracy: 0.9392
 152/1500 [==>...........................] - ETA: 10s - loss: 0.2203 - accuracy: 0.9394
 159/1500 [==>...........................] - ETA: 10s - loss: 0.2200 - accuracy: 0.9397
 166/1500 [==>...........................] - ETA: 10s - loss: 0.2240 - accuracy: 0.9388
 173/1500 [==>...........................] - ETA: 10s - loss: 0.2283 - accuracy: 0.9375
 179/1500 [==>...........................] - ETA: 10s - loss: 0.2279 - accuracy: 0.9380
 185/1500 [==>...........................] - ETA: 10s - loss: 0.2261 - accuracy: 0.9387
 192/1500 [==>...........................] - ETA: 10s - loss: 0.2241 - accuracy: 0.9388
 199/1500 [==>...........................] - ETA: 10s - loss: 0.2224 - accuracy: 0.9399
 204/1500 [===>..........................] - ETA: 10s - loss: 0.2226 - accuracy: 0.9398
 210/1500 [===>..........................] - ETA: 10s - loss: 0.2199 - accuracy: 0.9402
 217/1500 [===>..........................] - ETA: 10s - loss: 0.2229 - accuracy: 0.9394
 224/1500 [===>..........................] - ETA: 10s - loss: 0.2259 - accuracy: 0.9392
 231/1500 [===>..........................] - ETA: 10s - loss: 0.2229 - accuracy: 0.9401
 238/1500 [===>..........................] - ETA: 10s - loss: 0.2217 - accuracy: 0.9403
 244/1500 [===>..........................] - ETA: 10s - loss: 0.2211 - accuracy: 0.9402
 250/1500 [====>.........................] - ETA: 10s - loss: 0.2203 - accuracy: 0.9403
 257/1500 [====>.........................] - ETA: 10s - loss: 0.2198 - accuracy: 0.9401
 264/1500 [====>.........................] - ETA: 10s - loss: 0.2193 - accuracy: 0.9403
 271/1500 [====>.........................] - ETA: 9s - loss: 0.2175 - accuracy: 0.9410 
 278/1500 [====>.........................] - ETA: 9s - loss: 0.2180 - accuracy: 0.9403
 285/1500 [====>.........................] - ETA: 9s - loss: 0.2171 - accuracy: 0.9410
 292/1500 [====>.........................] - ETA: 9s - loss: 0.2163 - accuracy: 0.9411
 299/1500 [====>.........................] - ETA: 9s - loss: 0.2181 - accuracy: 0.9406
 306/1500 [=====>........................] - ETA: 9s - loss: 0.2178 - accuracy: 0.9406
 313/1500 [=====>........................] - ETA: 9s - loss: 0.2192 - accuracy: 0.9403
 320/1500 [=====>........................] - ETA: 9s - loss: 0.2184 - accuracy: 0.9404
 327/1500 [=====>........................] - ETA: 9s - loss: 0.2191 - accuracy: 0.9400
 334/1500 [=====>........................] - ETA: 9s - loss: 0.2196 - accuracy: 0.9397
 341/1500 [=====>........................] - ETA: 9s - loss: 0.2185 - accuracy: 0.9400
 348/1500 [=====>........................] - ETA: 9s - loss: 0.2184 - accuracy: 0.9402
 355/1500 [======>.......................] - ETA: 9s - loss: 0.2182 - accuracy: 0.9401
 362/1500 [======>.......................] - ETA: 9s - loss: 0.2188 - accuracy: 0.9397
 369/1500 [======>.......................] - ETA: 9s - loss: 0.2172 - accuracy: 0.9400
 376/1500 [======>.......................] - ETA: 9s - loss: 0.2161 - accuracy: 0.9405
 383/1500 [======>.......................] - ETA: 8s - loss: 0.2156 - accuracy: 0.9404
 390/1500 [======>.......................] - ETA: 8s - loss: 0.2166 - accuracy: 0.9400
 396/1500 [======>.......................] - ETA: 8s - loss: 0.2166 - accuracy: 0.9399
 402/1500 [=======>......................] - ETA: 8s - loss: 0.2167 - accuracy: 0.9398
 408/1500 [=======>......................] - ETA: 8s - loss: 0.2176 - accuracy: 0.9395
 414/1500 [=======>......................] - ETA: 8s - loss: 0.2170 - accuracy: 0.9396
 420/1500 [=======>......................] - ETA: 8s - loss: 0.2170 - accuracy: 0.9395
 427/1500 [=======>......................] - ETA: 8s - loss: 0.2167 - accuracy: 0.9397
 434/1500 [=======>......................] - ETA: 8s - loss: 0.2165 - accuracy: 0.9397
 438/1500 [=======>......................] - ETA: 8s - loss: 0.2162 - accuracy: 0.9397
 444/1500 [=======>......................] - ETA: 8s - loss: 0.2155 - accuracy: 0.9400
 450/1500 [========>.....................] - ETA: 8s - loss: 0.2158 - accuracy: 0.9398
 456/1500 [========>.....................] - ETA: 8s - loss: 0.2175 - accuracy: 0.9395
 461/1500 [========>.....................] - ETA: 8s - loss: 0.2176 - accuracy: 0.9397
 468/1500 [========>.....................] - ETA: 8s - loss: 0.2166 - accuracy: 0.9399
 474/1500 [========>.....................] - ETA: 8s - loss: 0.2164 - accuracy: 0.9401
 481/1500 [========>.....................] - ETA: 8s - loss: 0.2166 - accuracy: 0.9398
 487/1500 [========>.....................] - ETA: 8s - loss: 0.2178 - accuracy: 0.9397
 493/1500 [========>.....................] - ETA: 8s - loss: 0.2182 - accuracy: 0.9396
 499/1500 [========>.....................] - ETA: 8s - loss: 0.2190 - accuracy: 0.9392
 505/1500 [=========>....................] - ETA: 8s - loss: 0.2187 - accuracy: 0.9392
 509/1500 [=========>....................] - ETA: 8s - loss: 0.2185 - accuracy: 0.9392
 515/1500 [=========>....................] - ETA: 8s - loss: 0.2185 - accuracy: 0.9393
 521/1500 [=========>....................] - ETA: 8s - loss: 0.2180 - accuracy: 0.9395
 526/1500 [=========>....................] - ETA: 8s - loss: 0.2176 - accuracy: 0.9397
 531/1500 [=========>....................] - ETA: 8s - loss: 0.2173 - accuracy: 0.9399
 538/1500 [=========>....................] - ETA: 8s - loss: 0.2182 - accuracy: 0.9396
 545/1500 [=========>....................] - ETA: 7s - loss: 0.2177 - accuracy: 0.9400
 552/1500 [==========>...................] - ETA: 7s - loss: 0.2183 - accuracy: 0.9398
 559/1500 [==========>...................] - ETA: 7s - loss: 0.2186 - accuracy: 0.9396
 566/1500 [==========>...................] - ETA: 7s - loss: 0.2195 - accuracy: 0.9391
 573/1500 [==========>...................] - ETA: 7s - loss: 0.2201 - accuracy: 0.9390
 580/1500 [==========>...................] - ETA: 7s - loss: 0.2199 - accuracy: 0.9392
 587/1500 [==========>...................] - ETA: 7s - loss: 0.2198 - accuracy: 0.9390
 594/1500 [==========>...................] - ETA: 7s - loss: 0.2195 - accuracy: 0.9391
 601/1500 [===========>..................] - ETA: 7s - loss: 0.2197 - accuracy: 0.9392
 608/1500 [===========>..................] - ETA: 7s - loss: 0.2199 - accuracy: 0.9394
 615/1500 [===========>..................] - ETA: 7s - loss: 0.2195 - accuracy: 0.9396
 621/1500 [===========>..................] - ETA: 7s - loss: 0.2187 - accuracy: 0.9399
 627/1500 [===========>..................] - ETA: 7s - loss: 0.2182 - accuracy: 0.9400
 634/1500 [===========>..................] - ETA: 7s - loss: 0.2180 - accuracy: 0.9401
 641/1500 [===========>..................] - ETA: 7s - loss: 0.2191 - accuracy: 0.9397
 648/1500 [===========>..................] - ETA: 7s - loss: 0.2194 - accuracy: 0.9397
 655/1500 [============>.................] - ETA: 6s - loss: 0.2192 - accuracy: 0.9397
 662/1500 [============>.................] - ETA: 6s - loss: 0.2195 - accuracy: 0.9396
 669/1500 [============>.................] - ETA: 6s - loss: 0.2208 - accuracy: 0.9394
 676/1500 [============>.................] - ETA: 6s - loss: 0.2215 - accuracy: 0.9394
 683/1500 [============>.................] - ETA: 6s - loss: 0.2215 - accuracy: 0.9394
 690/1500 [============>.................] - ETA: 6s - loss: 0.2211 - accuracy: 0.9395
 697/1500 [============>.................] - ETA: 6s - loss: 0.2210 - accuracy: 0.9394
 705/1500 [=============>................] - ETA: 6s - loss: 0.2202 - accuracy: 0.9396
 712/1500 [=============>................] - ETA: 6s - loss: 0.2200 - accuracy: 0.9396
 719/1500 [=============>................] - ETA: 6s - loss: 0.2204 - accuracy: 0.9394
 726/1500 [=============>................] - ETA: 6s - loss: 0.2205 - accuracy: 0.9392
 733/1500 [=============>................] - ETA: 6s - loss: 0.2208 - accuracy: 0.9391
 740/1500 [=============>................] - ETA: 6s - loss: 0.2211 - accuracy: 0.9391
 747/1500 [=============>................] - ETA: 6s - loss: 0.2209 - accuracy: 0.9391
 754/1500 [==============>...............] - ETA: 6s - loss: 0.2208 - accuracy: 0.9392
 761/1500 [==============>...............] - ETA: 6s - loss: 0.2204 - accuracy: 0.9393
 768/1500 [==============>...............] - ETA: 5s - loss: 0.2209 - accuracy: 0.9389
 775/1500 [==============>...............] - ETA: 5s - loss: 0.2206 - accuracy: 0.9392
 782/1500 [==============>...............] - ETA: 5s - loss: 0.2211 - accuracy: 0.9392
 788/1500 [==============>...............] - ETA: 5s - loss: 0.2219 - accuracy: 0.9391
 795/1500 [==============>...............] - ETA: 5s - loss: 0.2218 - accuracy: 0.9392
 802/1500 [===============>..............] - ETA: 5s - loss: 0.2217 - accuracy: 0.9390
 809/1500 [===============>..............] - ETA: 5s - loss: 0.2213 - accuracy: 0.9392
 816/1500 [===============>..............] - ETA: 5s - loss: 0.2209 - accuracy: 0.9393
 822/1500 [===============>..............] - ETA: 5s - loss: 0.2211 - accuracy: 0.9394
 829/1500 [===============>..............] - ETA: 5s - loss: 0.2214 - accuracy: 0.9393
 832/1500 [===============>..............] - ETA: 5s - loss: 0.2218 - accuracy: 0.9392
 838/1500 [===============>..............] - ETA: 5s - loss: 0.2216 - accuracy: 0.9393
 845/1500 [===============>..............] - ETA: 5s - loss: 0.2218 - accuracy: 0.9393
 852/1500 [================>.............] - ETA: 5s - loss: 0.2221 - accuracy: 0.9393
 859/1500 [================>.............] - ETA: 5s - loss: 0.2222 - accuracy: 0.9394
 866/1500 [================>.............] - ETA: 5s - loss: 0.2219 - accuracy: 0.9394
 872/1500 [================>.............] - ETA: 5s - loss: 0.2222 - accuracy: 0.9393
 879/1500 [================>.............] - ETA: 5s - loss: 0.2221 - accuracy: 0.9393
 886/1500 [================>.............] - ETA: 5s - loss: 0.2223 - accuracy: 0.9392
 892/1500 [================>.............] - ETA: 4s - loss: 0.2222 - accuracy: 0.9392
 899/1500 [================>.............] - ETA: 4s - loss: 0.2218 - accuracy: 0.9394
 903/1500 [=================>............] - ETA: 4s - loss: 0.2217 - accuracy: 0.9395
 909/1500 [=================>............] - ETA: 4s - loss: 0.2214 - accuracy: 0.9396
 916/1500 [=================>............] - ETA: 4s - loss: 0.2213 - accuracy: 0.9396
 922/1500 [=================>............] - ETA: 4s - loss: 0.2213 - accuracy: 0.9396
 928/1500 [=================>............] - ETA: 4s - loss: 0.2216 - accuracy: 0.9395
 934/1500 [=================>............] - ETA: 4s - loss: 0.2216 - accuracy: 0.9393
 941/1500 [=================>............] - ETA: 4s - loss: 0.2212 - accuracy: 0.9394
 948/1500 [=================>............] - ETA: 4s - loss: 0.2215 - accuracy: 0.9393
 955/1500 [==================>...........] - ETA: 4s - loss: 0.2213 - accuracy: 0.9393
 962/1500 [==================>...........] - ETA: 4s - loss: 0.2215 - accuracy: 0.9394
 969/1500 [==================>...........] - ETA: 4s - loss: 0.2215 - accuracy: 0.9393
 976/1500 [==================>...........] - ETA: 4s - loss: 0.2214 - accuracy: 0.9395
 983/1500 [==================>...........] - ETA: 4s - loss: 0.2214 - accuracy: 0.9394
 990/1500 [==================>...........] - ETA: 4s - loss: 0.2208 - accuracy: 0.9396
 997/1500 [==================>...........] - ETA: 4s - loss: 0.2214 - accuracy: 0.9393
1004/1500 [===================>..........] - ETA: 4s - loss: 0.2212 - accuracy: 0.9394
1011/1500 [===================>..........] - ETA: 4s - loss: 0.2208 - accuracy: 0.9394
1018/1500 [===================>..........] - ETA: 3s - loss: 0.2210 - accuracy: 0.9393
1025/1500 [===================>..........] - ETA: 3s - loss: 0.2208 - accuracy: 0.9394
1032/1500 [===================>..........] - ETA: 3s - loss: 0.2211 - accuracy: 0.9393
1039/1500 [===================>..........] - ETA: 3s - loss: 0.2211 - accuracy: 0.9393
1046/1500 [===================>..........] - ETA: 3s - loss: 0.2214 - accuracy: 0.9393
1053/1500 [====================>.........] - ETA: 3s - loss: 0.2215 - accuracy: 0.9392
1060/1500 [====================>.........] - ETA: 3s - loss: 0.2213 - accuracy: 0.9393
1067/1500 [====================>.........] - ETA: 3s - loss: 0.2208 - accuracy: 0.9395
1073/1500 [====================>.........] - ETA: 3s - loss: 0.2207 - accuracy: 0.9395
1079/1500 [====================>.........] - ETA: 3s - loss: 0.2205 - accuracy: 0.9395
1086/1500 [====================>.........] - ETA: 3s - loss: 0.2205 - accuracy: 0.9394
1093/1500 [====================>.........] - ETA: 3s - loss: 0.2214 - accuracy: 0.9390
1099/1500 [====================>.........] - ETA: 3s - loss: 0.2212 - accuracy: 0.9390
1105/1500 [=====================>........] - ETA: 3s - loss: 0.2211 - accuracy: 0.9390
1112/1500 [=====================>........] - ETA: 3s - loss: 0.2215 - accuracy: 0.9390
1118/1500 [=====================>........] - ETA: 3s - loss: 0.2212 - accuracy: 0.9391
1125/1500 [=====================>........] - ETA: 3s - loss: 0.2211 - accuracy: 0.9392
1132/1500 [=====================>........] - ETA: 3s - loss: 0.2217 - accuracy: 0.9391
1139/1500 [=====================>........] - ETA: 2s - loss: 0.2218 - accuracy: 0.9390
1146/1500 [=====================>........] - ETA: 2s - loss: 0.2214 - accuracy: 0.9390
1153/1500 [======================>.......] - ETA: 2s - loss: 0.2212 - accuracy: 0.9390
1159/1500 [======================>.......] - ETA: 2s - loss: 0.2215 - accuracy: 0.9388
1166/1500 [======================>.......] - ETA: 2s - loss: 0.2214 - accuracy: 0.9389
1170/1500 [======================>.......] - ETA: 2s - loss: 0.2211 - accuracy: 0.9390
1176/1500 [======================>.......] - ETA: 2s - loss: 0.2213 - accuracy: 0.9390
1183/1500 [======================>.......] - ETA: 2s - loss: 0.2213 - accuracy: 0.9389
1190/1500 [======================>.......] - ETA: 2s - loss: 0.2211 - accuracy: 0.9390
1197/1500 [======================>.......] - ETA: 2s - loss: 0.2211 - accuracy: 0.9389
1204/1500 [=======================>......] - ETA: 2s - loss: 0.2213 - accuracy: 0.9388
1211/1500 [=======================>......] - ETA: 2s - loss: 0.2210 - accuracy: 0.9390
1218/1500 [=======================>......] - ETA: 2s - loss: 0.2209 - accuracy: 0.9390
1225/1500 [=======================>......] - ETA: 2s - loss: 0.2213 - accuracy: 0.9390
1232/1500 [=======================>......] - ETA: 2s - loss: 0.2211 - accuracy: 0.9390
1236/1500 [=======================>......] - ETA: 2s - loss: 0.2208 - accuracy: 0.9391
1243/1500 [=======================>......] - ETA: 2s - loss: 0.2208 - accuracy: 0.9391
1246/1500 [=======================>......] - ETA: 2s - loss: 0.2206 - accuracy: 0.9391
1248/1500 [=======================>......] - ETA: 2s - loss: 0.2209 - accuracy: 0.9390
1255/1500 [========================>.....] - ETA: 2s - loss: 0.2205 - accuracy: 0.9392
1262/1500 [========================>.....] - ETA: 1s - loss: 0.2202 - accuracy: 0.9392
1269/1500 [========================>.....] - ETA: 1s - loss: 0.2206 - accuracy: 0.9391
1276/1500 [========================>.....] - ETA: 1s - loss: 0.2206 - accuracy: 0.9391
1283/1500 [========================>.....] - ETA: 1s - loss: 0.2205 - accuracy: 0.9392
1289/1500 [========================>.....] - ETA: 1s - loss: 0.2204 - accuracy: 0.9392
1296/1500 [========================>.....] - ETA: 1s - loss: 0.2205 - accuracy: 0.9391
1303/1500 [=========================>....] - ETA: 1s - loss: 0.2207 - accuracy: 0.9391
1310/1500 [=========================>....] - ETA: 1s - loss: 0.2206 - accuracy: 0.9392
1316/1500 [=========================>....] - ETA: 1s - loss: 0.2205 - accuracy: 0.9392
1322/1500 [=========================>....] - ETA: 1s - loss: 0.2202 - accuracy: 0.9393
1328/1500 [=========================>....] - ETA: 1s - loss: 0.2200 - accuracy: 0.9394
1335/1500 [=========================>....] - ETA: 1s - loss: 0.2201 - accuracy: 0.9394
1341/1500 [=========================>....] - ETA: 1s - loss: 0.2201 - accuracy: 0.9393
1348/1500 [=========================>....] - ETA: 1s - loss: 0.2204 - accuracy: 0.9392
1355/1500 [==========================>...] - ETA: 1s - loss: 0.2201 - accuracy: 0.9393
1362/1500 [==========================>...] - ETA: 1s - loss: 0.2205 - accuracy: 0.9394
1369/1500 [==========================>...] - ETA: 1s - loss: 0.2205 - accuracy: 0.9395
1376/1500 [==========================>...] - ETA: 1s - loss: 0.2204 - accuracy: 0.9394
1382/1500 [==========================>...] - ETA: 0s - loss: 0.2204 - accuracy: 0.9394
1388/1500 [==========================>...] - ETA: 0s - loss: 0.2205 - accuracy: 0.9394
1394/1500 [==========================>...] - ETA: 0s - loss: 0.2208 - accuracy: 0.9394
1400/1500 [===========================>..] - ETA: 0s - loss: 0.2209 - accuracy: 0.9394
1407/1500 [===========================>..] - ETA: 0s - loss: 0.2212 - accuracy: 0.9394
1414/1500 [===========================>..] - ETA: 0s - loss: 0.2213 - accuracy: 0.9393
1420/1500 [===========================>..] - ETA: 0s - loss: 0.2211 - accuracy: 0.9394
1427/1500 [===========================>..] - ETA: 0s - loss: 0.2211 - accuracy: 0.9393
1434/1500 [===========================>..] - ETA: 0s - loss: 0.2213 - accuracy: 0.9393
1440/1500 [===========================>..] - ETA: 0s - loss: 0.2211 - accuracy: 0.9393
1447/1500 [===========================>..] - ETA: 0s - loss: 0.2211 - accuracy: 0.9393
1454/1500 [============================>.] - ETA: 0s - loss: 0.2210 - accuracy: 0.9394
1461/1500 [============================>.] - ETA: 0s - loss: 0.2209 - accuracy: 0.9394
1468/1500 [============================>.] - ETA: 0s - loss: 0.2209 - accuracy: 0.9394
1475/1500 [============================>.] - ETA: 0s - loss: 0.2207 - accuracy: 0.9394
1482/1500 [============================>.] - ETA: 0s - loss: 0.2210 - accuracy: 0.9393
1489/1500 [============================>.] - ETA: 0s - loss: 0.2211 - accuracy: 0.9392
1496/1500 [============================>.] - ETA: 0s - loss: 0.2213 - accuracy: 0.9391
1500/1500 [==============================] - 12s 8ms/step - loss: 0.2212 - accuracy: 0.9392

1500/1500 [==============================] - 15s 10ms/step - loss: 0.2212 - accuracy: 0.9392 - val_loss: 0.2163 - val_accuracy: 0.9381
Epoch 7/10

   1/1500 [..............................] - ETA: 1:29 - loss: 0.1731 - accuracy: 0.9688
   7/1500 [..............................] - ETA: 13s - loss: 0.2351 - accuracy: 0.9420 
  14/1500 [..............................] - ETA: 12s - loss: 0.1847 - accuracy: 0.9576
  21/1500 [..............................] - ETA: 12s - loss: 0.1820 - accuracy: 0.9554
  28/1500 [..............................] - ETA: 12s - loss: 0.1791 - accuracy: 0.9554
  35/1500 [..............................] - ETA: 12s - loss: 0.1795 - accuracy: 0.9518
  42/1500 [..............................] - ETA: 12s - loss: 0.1841 - accuracy: 0.9479
  49/1500 [..............................] - ETA: 12s - loss: 0.1926 - accuracy: 0.9458
  56/1500 [>.............................] - ETA: 11s - loss: 0.1898 - accuracy: 0.9464
  63/1500 [>.............................] - ETA: 11s - loss: 0.1843 - accuracy: 0.9484
  70/1500 [>.............................] - ETA: 11s - loss: 0.1949 - accuracy: 0.9460
  77/1500 [>.............................] - ETA: 11s - loss: 0.1950 - accuracy: 0.9460
  84/1500 [>.............................] - ETA: 11s - loss: 0.1981 - accuracy: 0.9468
  90/1500 [>.............................] - ETA: 11s - loss: 0.1943 - accuracy: 0.9486
  97/1500 [>.............................] - ETA: 11s - loss: 0.1937 - accuracy: 0.9494
 104/1500 [=>............................] - ETA: 11s - loss: 0.1923 - accuracy: 0.9498
 112/1500 [=>............................] - ETA: 11s - loss: 0.1920 - accuracy: 0.9501
 119/1500 [=>............................] - ETA: 11s - loss: 0.1920 - accuracy: 0.9501
 126/1500 [=>............................] - ETA: 10s - loss: 0.1910 - accuracy: 0.9504
 133/1500 [=>............................] - ETA: 10s - loss: 0.1914 - accuracy: 0.9504
 139/1500 [=>............................] - ETA: 10s - loss: 0.1926 - accuracy: 0.9499
 146/1500 [=>............................] - ETA: 10s - loss: 0.1909 - accuracy: 0.9495
 153/1500 [==>...........................] - ETA: 10s - loss: 0.1928 - accuracy: 0.9487
 160/1500 [==>...........................] - ETA: 10s - loss: 0.1935 - accuracy: 0.9488
 168/1500 [==>...........................] - ETA: 10s - loss: 0.1940 - accuracy: 0.9472
 175/1500 [==>...........................] - ETA: 10s - loss: 0.1942 - accuracy: 0.9475
 182/1500 [==>...........................] - ETA: 10s - loss: 0.1965 - accuracy: 0.9469
 189/1500 [==>...........................] - ETA: 10s - loss: 0.1989 - accuracy: 0.9459
 195/1500 [==>...........................] - ETA: 10s - loss: 0.1977 - accuracy: 0.9465
 201/1500 [===>..........................] - ETA: 10s - loss: 0.2014 - accuracy: 0.9453
 208/1500 [===>..........................] - ETA: 10s - loss: 0.2016 - accuracy: 0.9458
 213/1500 [===>..........................] - ETA: 10s - loss: 0.2026 - accuracy: 0.9456
 219/1500 [===>..........................] - ETA: 10s - loss: 0.2008 - accuracy: 0.9459
 225/1500 [===>..........................] - ETA: 10s - loss: 0.2006 - accuracy: 0.9456
 231/1500 [===>..........................] - ETA: 10s - loss: 0.2003 - accuracy: 0.9455
 237/1500 [===>..........................] - ETA: 10s - loss: 0.1990 - accuracy: 0.9461
 243/1500 [===>..........................] - ETA: 10s - loss: 0.1992 - accuracy: 0.9460
 250/1500 [====>.........................] - ETA: 10s - loss: 0.2001 - accuracy: 0.9454
 257/1500 [====>.........................] - ETA: 10s - loss: 0.2011 - accuracy: 0.9454
 264/1500 [====>.........................] - ETA: 10s - loss: 0.1999 - accuracy: 0.9455
 271/1500 [====>.........................] - ETA: 9s - loss: 0.1984 - accuracy: 0.9460 
 278/1500 [====>.........................] - ETA: 9s - loss: 0.1995 - accuracy: 0.9456
 284/1500 [====>.........................] - ETA: 9s - loss: 0.2008 - accuracy: 0.9449
 292/1500 [====>.........................] - ETA: 9s - loss: 0.2016 - accuracy: 0.9446
 299/1500 [====>.........................] - ETA: 9s - loss: 0.2024 - accuracy: 0.9441
 306/1500 [=====>........................] - ETA: 9s - loss: 0.2023 - accuracy: 0.9438
 313/1500 [=====>........................] - ETA: 9s - loss: 0.2023 - accuracy: 0.9440
 319/1500 [=====>........................] - ETA: 9s - loss: 0.2038 - accuracy: 0.9436
 326/1500 [=====>........................] - ETA: 9s - loss: 0.2034 - accuracy: 0.9435
 334/1500 [=====>........................] - ETA: 9s - loss: 0.2029 - accuracy: 0.9437
 341/1500 [=====>........................] - ETA: 9s - loss: 0.2018 - accuracy: 0.9441
 348/1500 [=====>........................] - ETA: 9s - loss: 0.2028 - accuracy: 0.9441
 355/1500 [======>.......................] - ETA: 9s - loss: 0.2050 - accuracy: 0.9432
 362/1500 [======>.......................] - ETA: 9s - loss: 0.2038 - accuracy: 0.9436
 369/1500 [======>.......................] - ETA: 9s - loss: 0.2038 - accuracy: 0.9436
 376/1500 [======>.......................] - ETA: 8s - loss: 0.2039 - accuracy: 0.9435
 383/1500 [======>.......................] - ETA: 8s - loss: 0.2051 - accuracy: 0.9431
 390/1500 [======>.......................] - ETA: 8s - loss: 0.2044 - accuracy: 0.9429
 397/1500 [======>.......................] - ETA: 8s - loss: 0.2033 - accuracy: 0.9434
 404/1500 [=======>......................] - ETA: 8s - loss: 0.2039 - accuracy: 0.9428
 411/1500 [=======>......................] - ETA: 8s - loss: 0.2031 - accuracy: 0.9429
 418/1500 [=======>......................] - ETA: 8s - loss: 0.2039 - accuracy: 0.9427
 425/1500 [=======>......................] - ETA: 8s - loss: 0.2043 - accuracy: 0.9426
 432/1500 [=======>......................] - ETA: 8s - loss: 0.2038 - accuracy: 0.9429
 439/1500 [=======>......................] - ETA: 8s - loss: 0.2037 - accuracy: 0.9426
 446/1500 [=======>......................] - ETA: 8s - loss: 0.2027 - accuracy: 0.9430
 453/1500 [========>.....................] - ETA: 8s - loss: 0.2016 - accuracy: 0.9436
 460/1500 [========>.....................] - ETA: 8s - loss: 0.2022 - accuracy: 0.9434
 467/1500 [========>.....................] - ETA: 8s - loss: 0.2018 - accuracy: 0.9435
 474/1500 [========>.....................] - ETA: 8s - loss: 0.2012 - accuracy: 0.9436
 481/1500 [========>.....................] - ETA: 8s - loss: 0.2021 - accuracy: 0.9434
 488/1500 [========>.....................] - ETA: 8s - loss: 0.2026 - accuracy: 0.9433
 495/1500 [========>.....................] - ETA: 7s - loss: 0.2026 - accuracy: 0.9434
 502/1500 [=========>....................] - ETA: 7s - loss: 0.2031 - accuracy: 0.9432
 509/1500 [=========>....................] - ETA: 7s - loss: 0.2032 - accuracy: 0.9430
 516/1500 [=========>....................] - ETA: 7s - loss: 0.2026 - accuracy: 0.9433
 523/1500 [=========>....................] - ETA: 7s - loss: 0.2027 - accuracy: 0.9434
 530/1500 [=========>....................] - ETA: 7s - loss: 0.2025 - accuracy: 0.9434
 537/1500 [=========>....................] - ETA: 7s - loss: 0.2022 - accuracy: 0.9434
 544/1500 [=========>....................] - ETA: 7s - loss: 0.2025 - accuracy: 0.9434
 551/1500 [==========>...................] - ETA: 7s - loss: 0.2044 - accuracy: 0.9431
 558/1500 [==========>...................] - ETA: 7s - loss: 0.2050 - accuracy: 0.9432
 565/1500 [==========>...................] - ETA: 7s - loss: 0.2045 - accuracy: 0.9433
 572/1500 [==========>...................] - ETA: 7s - loss: 0.2046 - accuracy: 0.9432
 579/1500 [==========>...................] - ETA: 7s - loss: 0.2040 - accuracy: 0.9433
 586/1500 [==========>...................] - ETA: 7s - loss: 0.2036 - accuracy: 0.9434
 593/1500 [==========>...................] - ETA: 7s - loss: 0.2045 - accuracy: 0.9432
 599/1500 [==========>...................] - ETA: 7s - loss: 0.2042 - accuracy: 0.9433
 606/1500 [===========>..................] - ETA: 7s - loss: 0.2044 - accuracy: 0.9432
 613/1500 [===========>..................] - ETA: 6s - loss: 0.2043 - accuracy: 0.9434
 618/1500 [===========>..................] - ETA: 6s - loss: 0.2046 - accuracy: 0.9434
 625/1500 [===========>..................] - ETA: 6s - loss: 0.2038 - accuracy: 0.9437
 631/1500 [===========>..................] - ETA: 6s - loss: 0.2042 - accuracy: 0.9435
 638/1500 [===========>..................] - ETA: 6s - loss: 0.2040 - accuracy: 0.9435
 644/1500 [===========>..................] - ETA: 6s - loss: 0.2046 - accuracy: 0.9431
 651/1500 [============>.................] - ETA: 6s - loss: 0.2051 - accuracy: 0.9431
 658/1500 [============>.................] - ETA: 6s - loss: 0.2054 - accuracy: 0.9431
 664/1500 [============>.................] - ETA: 6s - loss: 0.2059 - accuracy: 0.9428
 671/1500 [============>.................] - ETA: 6s - loss: 0.2058 - accuracy: 0.9429
 678/1500 [============>.................] - ETA: 6s - loss: 0.2059 - accuracy: 0.9428
 684/1500 [============>.................] - ETA: 6s - loss: 0.2057 - accuracy: 0.9430
 691/1500 [============>.................] - ETA: 6s - loss: 0.2060 - accuracy: 0.9431
 698/1500 [============>.................] - ETA: 6s - loss: 0.2055 - accuracy: 0.9432
 705/1500 [=============>................] - ETA: 6s - loss: 0.2063 - accuracy: 0.9433
 712/1500 [=============>................] - ETA: 6s - loss: 0.2061 - accuracy: 0.9434
 719/1500 [=============>................] - ETA: 6s - loss: 0.2066 - accuracy: 0.9433
 726/1500 [=============>................] - ETA: 6s - loss: 0.2064 - accuracy: 0.9433
 734/1500 [=============>................] - ETA: 6s - loss: 0.2071 - accuracy: 0.9433
 742/1500 [=============>................] - ETA: 5s - loss: 0.2073 - accuracy: 0.9432
 750/1500 [==============>...............] - ETA: 5s - loss: 0.2066 - accuracy: 0.9435
 757/1500 [==============>...............] - ETA: 5s - loss: 0.2065 - accuracy: 0.9433
 764/1500 [==============>...............] - ETA: 5s - loss: 0.2063 - accuracy: 0.9433
 771/1500 [==============>...............] - ETA: 5s - loss: 0.2062 - accuracy: 0.9432
 778/1500 [==============>...............] - ETA: 5s - loss: 0.2059 - accuracy: 0.9433
 785/1500 [==============>...............] - ETA: 5s - loss: 0.2056 - accuracy: 0.9435
 792/1500 [==============>...............] - ETA: 5s - loss: 0.2054 - accuracy: 0.9433
 800/1500 [===============>..............] - ETA: 5s - loss: 0.2053 - accuracy: 0.9434
 808/1500 [===============>..............] - ETA: 5s - loss: 0.2051 - accuracy: 0.9435
 816/1500 [===============>..............] - ETA: 5s - loss: 0.2051 - accuracy: 0.9436
 824/1500 [===============>..............] - ETA: 5s - loss: 0.2049 - accuracy: 0.9435
 832/1500 [===============>..............] - ETA: 5s - loss: 0.2053 - accuracy: 0.9434
 839/1500 [===============>..............] - ETA: 5s - loss: 0.2050 - accuracy: 0.9435
 846/1500 [===============>..............] - ETA: 5s - loss: 0.2047 - accuracy: 0.9436
 853/1500 [================>.............] - ETA: 5s - loss: 0.2045 - accuracy: 0.9437
 858/1500 [================>.............] - ETA: 5s - loss: 0.2051 - accuracy: 0.9436
 864/1500 [================>.............] - ETA: 4s - loss: 0.2052 - accuracy: 0.9435
 870/1500 [================>.............] - ETA: 4s - loss: 0.2050 - accuracy: 0.9436
 876/1500 [================>.............] - ETA: 4s - loss: 0.2047 - accuracy: 0.9436
 883/1500 [================>.............] - ETA: 4s - loss: 0.2049 - accuracy: 0.9435
 890/1500 [================>.............] - ETA: 4s - loss: 0.2051 - accuracy: 0.9435
 897/1500 [================>.............] - ETA: 4s - loss: 0.2048 - accuracy: 0.9436
 904/1500 [=================>............] - ETA: 4s - loss: 0.2053 - accuracy: 0.9436
 911/1500 [=================>............] - ETA: 4s - loss: 0.2054 - accuracy: 0.9437
 918/1500 [=================>............] - ETA: 4s - loss: 0.2054 - accuracy: 0.9436
 925/1500 [=================>............] - ETA: 4s - loss: 0.2057 - accuracy: 0.9435
 932/1500 [=================>............] - ETA: 4s - loss: 0.2060 - accuracy: 0.9435
 939/1500 [=================>............] - ETA: 4s - loss: 0.2063 - accuracy: 0.9435
 946/1500 [=================>............] - ETA: 4s - loss: 0.2064 - accuracy: 0.9434
 952/1500 [==================>...........] - ETA: 4s - loss: 0.2065 - accuracy: 0.9434
 959/1500 [==================>...........] - ETA: 4s - loss: 0.2065 - accuracy: 0.9433
 966/1500 [==================>...........] - ETA: 4s - loss: 0.2067 - accuracy: 0.9433
 973/1500 [==================>...........] - ETA: 4s - loss: 0.2069 - accuracy: 0.9432
 980/1500 [==================>...........] - ETA: 4s - loss: 0.2066 - accuracy: 0.9433
 987/1500 [==================>...........] - ETA: 4s - loss: 0.2061 - accuracy: 0.9435
 993/1500 [==================>...........] - ETA: 3s - loss: 0.2058 - accuracy: 0.9435
1000/1500 [===================>..........] - ETA: 3s - loss: 0.2055 - accuracy: 0.9436
1007/1500 [===================>..........] - ETA: 3s - loss: 0.2055 - accuracy: 0.9436
1014/1500 [===================>..........] - ETA: 3s - loss: 0.2056 - accuracy: 0.9437
1021/1500 [===================>..........] - ETA: 3s - loss: 0.2056 - accuracy: 0.9437
1028/1500 [===================>..........] - ETA: 3s - loss: 0.2056 - accuracy: 0.9436
1034/1500 [===================>..........] - ETA: 3s - loss: 0.2059 - accuracy: 0.9435
1040/1500 [===================>..........] - ETA: 3s - loss: 0.2060 - accuracy: 0.9436
1046/1500 [===================>..........] - ETA: 3s - loss: 0.2061 - accuracy: 0.9435
1053/1500 [====================>.........] - ETA: 3s - loss: 0.2062 - accuracy: 0.9433
1059/1500 [====================>.........] - ETA: 3s - loss: 0.2063 - accuracy: 0.9433
1066/1500 [====================>.........] - ETA: 3s - loss: 0.2066 - accuracy: 0.9432
1072/1500 [====================>.........] - ETA: 3s - loss: 0.2062 - accuracy: 0.9433
1078/1500 [====================>.........] - ETA: 3s - loss: 0.2063 - accuracy: 0.9433
1085/1500 [====================>.........] - ETA: 3s - loss: 0.2059 - accuracy: 0.9435
1092/1500 [====================>.........] - ETA: 3s - loss: 0.2061 - accuracy: 0.9433
1098/1500 [====================>.........] - ETA: 3s - loss: 0.2059 - accuracy: 0.9435
1104/1500 [=====================>........] - ETA: 3s - loss: 0.2059 - accuracy: 0.9434
1110/1500 [=====================>........] - ETA: 3s - loss: 0.2056 - accuracy: 0.9435
1117/1500 [=====================>........] - ETA: 3s - loss: 0.2053 - accuracy: 0.9436
1123/1500 [=====================>........] - ETA: 2s - loss: 0.2052 - accuracy: 0.9436
1126/1500 [=====================>........] - ETA: 2s - loss: 0.2055 - accuracy: 0.9436
1132/1500 [=====================>........] - ETA: 2s - loss: 0.2054 - accuracy: 0.9437
1138/1500 [=====================>........] - ETA: 2s - loss: 0.2055 - accuracy: 0.9437
1145/1500 [=====================>........] - ETA: 2s - loss: 0.2055 - accuracy: 0.9437
1152/1500 [======================>.......] - ETA: 2s - loss: 0.2054 - accuracy: 0.9436
1157/1500 [======================>.......] - ETA: 2s - loss: 0.2056 - accuracy: 0.9436
1164/1500 [======================>.......] - ETA: 2s - loss: 0.2053 - accuracy: 0.9436
1171/1500 [======================>.......] - ETA: 2s - loss: 0.2054 - accuracy: 0.9436
1174/1500 [======================>.......] - ETA: 2s - loss: 0.2053 - accuracy: 0.9436
1181/1500 [======================>.......] - ETA: 2s - loss: 0.2052 - accuracy: 0.9438
1187/1500 [======================>.......] - ETA: 2s - loss: 0.2048 - accuracy: 0.9439
1193/1500 [======================>.......] - ETA: 2s - loss: 0.2051 - accuracy: 0.9438
1200/1500 [=======================>......] - ETA: 2s - loss: 0.2053 - accuracy: 0.9438
1207/1500 [=======================>......] - ETA: 2s - loss: 0.2054 - accuracy: 0.9438
1213/1500 [=======================>......] - ETA: 2s - loss: 0.2057 - accuracy: 0.9438
1220/1500 [=======================>......] - ETA: 2s - loss: 0.2056 - accuracy: 0.9438
1227/1500 [=======================>......] - ETA: 2s - loss: 0.2057 - accuracy: 0.9438
1234/1500 [=======================>......] - ETA: 2s - loss: 0.2058 - accuracy: 0.9437
1240/1500 [=======================>......] - ETA: 2s - loss: 0.2056 - accuracy: 0.9438
1247/1500 [=======================>......] - ETA: 2s - loss: 0.2054 - accuracy: 0.9438
1254/1500 [========================>.....] - ETA: 1s - loss: 0.2057 - accuracy: 0.9436
1261/1500 [========================>.....] - ETA: 1s - loss: 0.2059 - accuracy: 0.9435
1268/1500 [========================>.....] - ETA: 1s - loss: 0.2065 - accuracy: 0.9433
1274/1500 [========================>.....] - ETA: 1s - loss: 0.2069 - accuracy: 0.9432
1281/1500 [========================>.....] - ETA: 1s - loss: 0.2067 - accuracy: 0.9432
1287/1500 [========================>.....] - ETA: 1s - loss: 0.2068 - accuracy: 0.9432
1294/1500 [========================>.....] - ETA: 1s - loss: 0.2064 - accuracy: 0.9433
1301/1500 [=========================>....] - ETA: 1s - loss: 0.2066 - accuracy: 0.9432
1308/1500 [=========================>....] - ETA: 1s - loss: 0.2066 - accuracy: 0.9433
1315/1500 [=========================>....] - ETA: 1s - loss: 0.2068 - accuracy: 0.9432
1322/1500 [=========================>....] - ETA: 1s - loss: 0.2067 - accuracy: 0.9431
1329/1500 [=========================>....] - ETA: 1s - loss: 0.2067 - accuracy: 0.9431
1336/1500 [=========================>....] - ETA: 1s - loss: 0.2065 - accuracy: 0.9431
1343/1500 [=========================>....] - ETA: 1s - loss: 0.2062 - accuracy: 0.9432
1350/1500 [==========================>...] - ETA: 1s - loss: 0.2061 - accuracy: 0.9432
1357/1500 [==========================>...] - ETA: 1s - loss: 0.2059 - accuracy: 0.9433
1364/1500 [==========================>...] - ETA: 1s - loss: 0.2063 - accuracy: 0.9431
1371/1500 [==========================>...] - ETA: 1s - loss: 0.2062 - accuracy: 0.9432
1377/1500 [==========================>...] - ETA: 0s - loss: 0.2059 - accuracy: 0.9433
1383/1500 [==========================>...] - ETA: 0s - loss: 0.2059 - accuracy: 0.9434
1390/1500 [==========================>...] - ETA: 0s - loss: 0.2059 - accuracy: 0.9433
1397/1500 [==========================>...] - ETA: 0s - loss: 0.2062 - accuracy: 0.9433
1404/1500 [===========================>..] - ETA: 0s - loss: 0.2062 - accuracy: 0.9434
1411/1500 [===========================>..] - ETA: 0s - loss: 0.2058 - accuracy: 0.9435
1418/1500 [===========================>..] - ETA: 0s - loss: 0.2057 - accuracy: 0.9435
1424/1500 [===========================>..] - ETA: 0s - loss: 0.2056 - accuracy: 0.9434
1431/1500 [===========================>..] - ETA: 0s - loss: 0.2060 - accuracy: 0.9433
1438/1500 [===========================>..] - ETA: 0s - loss: 0.2061 - accuracy: 0.9433
1445/1500 [===========================>..] - ETA: 0s - loss: 0.2057 - accuracy: 0.9434
1452/1500 [============================>.] - ETA: 0s - loss: 0.2057 - accuracy: 0.9434
1459/1500 [============================>.] - ETA: 0s - loss: 0.2055 - accuracy: 0.9435
1465/1500 [============================>.] - ETA: 0s - loss: 0.2055 - accuracy: 0.9435
1472/1500 [============================>.] - ETA: 0s - loss: 0.2054 - accuracy: 0.9435
1478/1500 [============================>.] - ETA: 0s - loss: 0.2052 - accuracy: 0.9435
1485/1500 [============================>.] - ETA: 0s - loss: 0.2051 - accuracy: 0.9436
1491/1500 [============================>.] - ETA: 0s - loss: 0.2051 - accuracy: 0.9436
1497/1500 [============================>.] - ETA: 0s - loss: 0.2051 - accuracy: 0.9436
1500/1500 [==============================] - 12s 8ms/step - loss: 0.2049 - accuracy: 0.9437

1500/1500 [==============================] - 15s 10ms/step - loss: 0.2049 - accuracy: 0.9437 - val_loss: 0.2022 - val_accuracy: 0.9447
Epoch 8/10

   1/1500 [..............................] - ETA: 27s - loss: 0.1610 - accuracy: 0.9688
   8/1500 [..............................] - ETA: 10s - loss: 0.1887 - accuracy: 0.9531
  15/1500 [..............................] - ETA: 10s - loss: 0.1924 - accuracy: 0.9438
  22/1500 [..............................] - ETA: 10s - loss: 0.1756 - accuracy: 0.9545
  29/1500 [..............................] - ETA: 11s - loss: 0.1773 - accuracy: 0.9547
  35/1500 [..............................] - ETA: 11s - loss: 0.2061 - accuracy: 0.9482
  41/1500 [..............................] - ETA: 11s - loss: 0.2111 - accuracy: 0.9497
  48/1500 [..............................] - ETA: 11s - loss: 0.2069 - accuracy: 0.9486
  55/1500 [>.............................] - ETA: 11s - loss: 0.2106 - accuracy: 0.9477
  62/1500 [>.............................] - ETA: 11s - loss: 0.2136 - accuracy: 0.9466
  69/1500 [>.............................] - ETA: 11s - loss: 0.2078 - accuracy: 0.9461
  76/1500 [>.............................] - ETA: 11s - loss: 0.2048 - accuracy: 0.9465
  83/1500 [>.............................] - ETA: 11s - loss: 0.2000 - accuracy: 0.9480
  89/1500 [>.............................] - ETA: 11s - loss: 0.2001 - accuracy: 0.9477
  95/1500 [>.............................] - ETA: 11s - loss: 0.2005 - accuracy: 0.9487
 102/1500 [=>............................] - ETA: 11s - loss: 0.2019 - accuracy: 0.9470
 109/1500 [=>............................] - ETA: 11s - loss: 0.2007 - accuracy: 0.9475
 117/1500 [=>............................] - ETA: 10s - loss: 0.2043 - accuracy: 0.9455
 124/1500 [=>............................] - ETA: 10s - loss: 0.2019 - accuracy: 0.9466
 131/1500 [=>............................] - ETA: 10s - loss: 0.1990 - accuracy: 0.9478
 138/1500 [=>............................] - ETA: 10s - loss: 0.1997 - accuracy: 0.9470
 145/1500 [=>............................] - ETA: 10s - loss: 0.2013 - accuracy: 0.9463
 151/1500 [==>...........................] - ETA: 10s - loss: 0.2023 - accuracy: 0.9468
 159/1500 [==>...........................] - ETA: 10s - loss: 0.1991 - accuracy: 0.9485
 164/1500 [==>...........................] - ETA: 10s - loss: 0.2033 - accuracy: 0.9476
 169/1500 [==>...........................] - ETA: 11s - loss: 0.2000 - accuracy: 0.9488
 175/1500 [==>...........................] - ETA: 11s - loss: 0.1985 - accuracy: 0.9495
 182/1500 [==>...........................] - ETA: 10s - loss: 0.1980 - accuracy: 0.9495
 189/1500 [==>...........................] - ETA: 10s - loss: 0.1992 - accuracy: 0.9486
 196/1500 [==>...........................] - ETA: 10s - loss: 0.2004 - accuracy: 0.9482
 203/1500 [===>..........................] - ETA: 10s - loss: 0.1996 - accuracy: 0.9480
 210/1500 [===>..........................] - ETA: 10s - loss: 0.1975 - accuracy: 0.9485
 215/1500 [===>..........................] - ETA: 10s - loss: 0.1981 - accuracy: 0.9487
 222/1500 [===>..........................] - ETA: 10s - loss: 0.1982 - accuracy: 0.9486
 228/1500 [===>..........................] - ETA: 10s - loss: 0.1992 - accuracy: 0.9482
 235/1500 [===>..........................] - ETA: 10s - loss: 0.2007 - accuracy: 0.9476
 241/1500 [===>..........................] - ETA: 10s - loss: 0.1990 - accuracy: 0.9479
 247/1500 [===>..........................] - ETA: 10s - loss: 0.1977 - accuracy: 0.9483
 253/1500 [====>.........................] - ETA: 10s - loss: 0.1983 - accuracy: 0.9479
 258/1500 [====>.........................] - ETA: 10s - loss: 0.1973 - accuracy: 0.9483
 265/1500 [====>.........................] - ETA: 10s - loss: 0.1982 - accuracy: 0.9475
 272/1500 [====>.........................] - ETA: 10s - loss: 0.1994 - accuracy: 0.9467
 279/1500 [====>.........................] - ETA: 10s - loss: 0.1994 - accuracy: 0.9465
 286/1500 [====>.........................] - ETA: 10s - loss: 0.1996 - accuracy: 0.9461
 293/1500 [====>.........................] - ETA: 10s - loss: 0.1995 - accuracy: 0.9461
 300/1500 [=====>........................] - ETA: 10s - loss: 0.1989 - accuracy: 0.9459
 307/1500 [=====>........................] - ETA: 10s - loss: 0.1985 - accuracy: 0.9461
 314/1500 [=====>........................] - ETA: 9s - loss: 0.1986 - accuracy: 0.9463 
 321/1500 [=====>........................] - ETA: 9s - loss: 0.1995 - accuracy: 0.9458
 328/1500 [=====>........................] - ETA: 9s - loss: 0.1994 - accuracy: 0.9460
 335/1500 [=====>........................] - ETA: 9s - loss: 0.1992 - accuracy: 0.9459
 342/1500 [=====>........................] - ETA: 9s - loss: 0.1999 - accuracy: 0.9458
 349/1500 [=====>........................] - ETA: 9s - loss: 0.2003 - accuracy: 0.9456
 356/1500 [======>.......................] - ETA: 9s - loss: 0.1996 - accuracy: 0.9458
 363/1500 [======>.......................] - ETA: 9s - loss: 0.1992 - accuracy: 0.9459
 370/1500 [======>.......................] - ETA: 9s - loss: 0.1990 - accuracy: 0.9460
 377/1500 [======>.......................] - ETA: 9s - loss: 0.1987 - accuracy: 0.9460
 384/1500 [======>.......................] - ETA: 9s - loss: 0.1981 - accuracy: 0.9462
 391/1500 [======>.......................] - ETA: 9s - loss: 0.1982 - accuracy: 0.9462
 398/1500 [======>.......................] - ETA: 9s - loss: 0.1974 - accuracy: 0.9464
 405/1500 [=======>......................] - ETA: 9s - loss: 0.1972 - accuracy: 0.9464
 412/1500 [=======>......................] - ETA: 9s - loss: 0.1972 - accuracy: 0.9465
 419/1500 [=======>......................] - ETA: 9s - loss: 0.1966 - accuracy: 0.9464
 425/1500 [=======>......................] - ETA: 8s - loss: 0.1970 - accuracy: 0.9465
 431/1500 [=======>......................] - ETA: 8s - loss: 0.1964 - accuracy: 0.9467
 437/1500 [=======>......................] - ETA: 8s - loss: 0.1962 - accuracy: 0.9467
 444/1500 [=======>......................] - ETA: 8s - loss: 0.1966 - accuracy: 0.9466
 450/1500 [========>.....................] - ETA: 8s - loss: 0.1968 - accuracy: 0.9465
 457/1500 [========>.....................] - ETA: 8s - loss: 0.1953 - accuracy: 0.9471
 464/1500 [========>.....................] - ETA: 8s - loss: 0.1952 - accuracy: 0.9472
 471/1500 [========>.....................] - ETA: 8s - loss: 0.1955 - accuracy: 0.9469
 478/1500 [========>.....................] - ETA: 8s - loss: 0.1957 - accuracy: 0.9467
 484/1500 [========>.....................] - ETA: 8s - loss: 0.1948 - accuracy: 0.9470
 491/1500 [========>.....................] - ETA: 8s - loss: 0.1940 - accuracy: 0.9472
 498/1500 [========>.....................] - ETA: 8s - loss: 0.1949 - accuracy: 0.9472
 505/1500 [=========>....................] - ETA: 8s - loss: 0.1948 - accuracy: 0.9472
 512/1500 [=========>....................] - ETA: 8s - loss: 0.1946 - accuracy: 0.9473
 518/1500 [=========>....................] - ETA: 8s - loss: 0.1939 - accuracy: 0.9474
 525/1500 [=========>....................] - ETA: 8s - loss: 0.1935 - accuracy: 0.9475
 532/1500 [=========>....................] - ETA: 8s - loss: 0.1929 - accuracy: 0.9477
 539/1500 [=========>....................] - ETA: 7s - loss: 0.1934 - accuracy: 0.9473
 546/1500 [=========>....................] - ETA: 7s - loss: 0.1931 - accuracy: 0.9476
 553/1500 [==========>...................] - ETA: 7s - loss: 0.1926 - accuracy: 0.9478
 560/1500 [==========>...................] - ETA: 7s - loss: 0.1921 - accuracy: 0.9479
 567/1500 [==========>...................] - ETA: 7s - loss: 0.1926 - accuracy: 0.9479
 574/1500 [==========>...................] - ETA: 7s - loss: 0.1926 - accuracy: 0.9479
 581/1500 [==========>...................] - ETA: 7s - loss: 0.1922 - accuracy: 0.9481
 588/1500 [==========>...................] - ETA: 7s - loss: 0.1917 - accuracy: 0.9482
 595/1500 [==========>...................] - ETA: 7s - loss: 0.1911 - accuracy: 0.9484
 601/1500 [===========>..................] - ETA: 7s - loss: 0.1909 - accuracy: 0.9483
 608/1500 [===========>..................] - ETA: 7s - loss: 0.1922 - accuracy: 0.9480
 615/1500 [===========>..................] - ETA: 7s - loss: 0.1927 - accuracy: 0.9478
 622/1500 [===========>..................] - ETA: 7s - loss: 0.1924 - accuracy: 0.9477
 629/1500 [===========>..................] - ETA: 7s - loss: 0.1920 - accuracy: 0.9478
 636/1500 [===========>..................] - ETA: 7s - loss: 0.1921 - accuracy: 0.9477
 643/1500 [===========>..................] - ETA: 7s - loss: 0.1923 - accuracy: 0.9478
 651/1500 [============>.................] - ETA: 6s - loss: 0.1922 - accuracy: 0.9478
 658/1500 [============>.................] - ETA: 6s - loss: 0.1928 - accuracy: 0.9476
 663/1500 [============>.................] - ETA: 6s - loss: 0.1929 - accuracy: 0.9476
 669/1500 [============>.................] - ETA: 6s - loss: 0.1928 - accuracy: 0.9477
 675/1500 [============>.................] - ETA: 6s - loss: 0.1936 - accuracy: 0.9475
 682/1500 [============>.................] - ETA: 6s - loss: 0.1933 - accuracy: 0.9475
 688/1500 [============>.................] - ETA: 6s - loss: 0.1933 - accuracy: 0.9475
 695/1500 [============>.................] - ETA: 6s - loss: 0.1932 - accuracy: 0.9476
 701/1500 [=============>................] - ETA: 6s - loss: 0.1933 - accuracy: 0.9476
 708/1500 [=============>................] - ETA: 6s - loss: 0.1930 - accuracy: 0.9477
 714/1500 [=============>................] - ETA: 6s - loss: 0.1927 - accuracy: 0.9478
 720/1500 [=============>................] - ETA: 6s - loss: 0.1926 - accuracy: 0.9477
 727/1500 [=============>................] - ETA: 6s - loss: 0.1926 - accuracy: 0.9476
 733/1500 [=============>................] - ETA: 6s - loss: 0.1921 - accuracy: 0.9479
 739/1500 [=============>................] - ETA: 6s - loss: 0.1920 - accuracy: 0.9479
 745/1500 [=============>................] - ETA: 6s - loss: 0.1918 - accuracy: 0.9481
 751/1500 [==============>...............] - ETA: 6s - loss: 0.1920 - accuracy: 0.9479
 758/1500 [==============>...............] - ETA: 6s - loss: 0.1917 - accuracy: 0.9478
 765/1500 [==============>...............] - ETA: 6s - loss: 0.1917 - accuracy: 0.9479
 771/1500 [==============>...............] - ETA: 6s - loss: 0.1918 - accuracy: 0.9478
 778/1500 [==============>...............] - ETA: 5s - loss: 0.1924 - accuracy: 0.9476
 784/1500 [==============>...............] - ETA: 5s - loss: 0.1932 - accuracy: 0.9473
 790/1500 [==============>...............] - ETA: 5s - loss: 0.1930 - accuracy: 0.9473
 797/1500 [==============>...............] - ETA: 5s - loss: 0.1933 - accuracy: 0.9472
 803/1500 [===============>..............] - ETA: 5s - loss: 0.1933 - accuracy: 0.9472
 810/1500 [===============>..............] - ETA: 5s - loss: 0.1930 - accuracy: 0.9473
 814/1500 [===============>..............] - ETA: 5s - loss: 0.1931 - accuracy: 0.9471
 821/1500 [===============>..............] - ETA: 5s - loss: 0.1936 - accuracy: 0.9470
 827/1500 [===============>..............] - ETA: 5s - loss: 0.1932 - accuracy: 0.9471
 833/1500 [===============>..............] - ETA: 5s - loss: 0.1933 - accuracy: 0.9471
 839/1500 [===============>..............] - ETA: 5s - loss: 0.1938 - accuracy: 0.9470
 846/1500 [===============>..............] - ETA: 5s - loss: 0.1945 - accuracy: 0.9467
 852/1500 [================>.............] - ETA: 5s - loss: 0.1947 - accuracy: 0.9466
 858/1500 [================>.............] - ETA: 5s - loss: 0.1945 - accuracy: 0.9466
 865/1500 [================>.............] - ETA: 5s - loss: 0.1943 - accuracy: 0.9467
 872/1500 [================>.............] - ETA: 5s - loss: 0.1948 - accuracy: 0.9464
 879/1500 [================>.............] - ETA: 5s - loss: 0.1946 - accuracy: 0.9463
 886/1500 [================>.............] - ETA: 5s - loss: 0.1947 - accuracy: 0.9464
 892/1500 [================>.............] - ETA: 5s - loss: 0.1945 - accuracy: 0.9463
 898/1500 [================>.............] - ETA: 4s - loss: 0.1943 - accuracy: 0.9464
 905/1500 [=================>............] - ETA: 4s - loss: 0.1940 - accuracy: 0.9465
 911/1500 [=================>............] - ETA: 4s - loss: 0.1941 - accuracy: 0.9463
 918/1500 [=================>............] - ETA: 4s - loss: 0.1938 - accuracy: 0.9464
 925/1500 [=================>............] - ETA: 4s - loss: 0.1938 - accuracy: 0.9464
 932/1500 [=================>............] - ETA: 4s - loss: 0.1941 - accuracy: 0.9464
 939/1500 [=================>............] - ETA: 4s - loss: 0.1941 - accuracy: 0.9463
 946/1500 [=================>............] - ETA: 4s - loss: 0.1942 - accuracy: 0.9462
 952/1500 [==================>...........] - ETA: 4s - loss: 0.1938 - accuracy: 0.9463
 958/1500 [==================>...........] - ETA: 4s - loss: 0.1937 - accuracy: 0.9462
 965/1500 [==================>...........] - ETA: 4s - loss: 0.1937 - accuracy: 0.9461
 971/1500 [==================>...........] - ETA: 4s - loss: 0.1940 - accuracy: 0.9460
 977/1500 [==================>...........] - ETA: 4s - loss: 0.1941 - accuracy: 0.9458
 984/1500 [==================>...........] - ETA: 4s - loss: 0.1941 - accuracy: 0.9459
 991/1500 [==================>...........] - ETA: 4s - loss: 0.1938 - accuracy: 0.9460
 998/1500 [==================>...........] - ETA: 4s - loss: 0.1936 - accuracy: 0.9460
1005/1500 [===================>..........] - ETA: 4s - loss: 0.1936 - accuracy: 0.9460
1012/1500 [===================>..........] - ETA: 4s - loss: 0.1934 - accuracy: 0.9461
1019/1500 [===================>..........] - ETA: 3s - loss: 0.1929 - accuracy: 0.9462
1026/1500 [===================>..........] - ETA: 3s - loss: 0.1930 - accuracy: 0.9461
1033/1500 [===================>..........] - ETA: 3s - loss: 0.1934 - accuracy: 0.9462
1040/1500 [===================>..........] - ETA: 3s - loss: 0.1931 - accuracy: 0.9462
1046/1500 [===================>..........] - ETA: 3s - loss: 0.1931 - accuracy: 0.9462
1053/1500 [====================>.........] - ETA: 3s - loss: 0.1929 - accuracy: 0.9463
1059/1500 [====================>.........] - ETA: 3s - loss: 0.1927 - accuracy: 0.9464
1066/1500 [====================>.........] - ETA: 3s - loss: 0.1926 - accuracy: 0.9464
1073/1500 [====================>.........] - ETA: 3s - loss: 0.1927 - accuracy: 0.9463
1080/1500 [====================>.........] - ETA: 3s - loss: 0.1924 - accuracy: 0.9464
1087/1500 [====================>.........] - ETA: 3s - loss: 0.1923 - accuracy: 0.9464
1094/1500 [====================>.........] - ETA: 3s - loss: 0.1920 - accuracy: 0.9465
1101/1500 [=====================>........] - ETA: 3s - loss: 0.1922 - accuracy: 0.9465
1108/1500 [=====================>........] - ETA: 3s - loss: 0.1924 - accuracy: 0.9464
1115/1500 [=====================>........] - ETA: 3s - loss: 0.1923 - accuracy: 0.9464
1122/1500 [=====================>........] - ETA: 3s - loss: 0.1923 - accuracy: 0.9464
1129/1500 [=====================>........] - ETA: 3s - loss: 0.1925 - accuracy: 0.9464
1136/1500 [=====================>........] - ETA: 2s - loss: 0.1923 - accuracy: 0.9464
1143/1500 [=====================>........] - ETA: 2s - loss: 0.1928 - accuracy: 0.9462
1150/1500 [======================>.......] - ETA: 2s - loss: 0.1926 - accuracy: 0.9463
1157/1500 [======================>.......] - ETA: 2s - loss: 0.1926 - accuracy: 0.9463
1164/1500 [======================>.......] - ETA: 2s - loss: 0.1927 - accuracy: 0.9463
1170/1500 [======================>.......] - ETA: 2s - loss: 0.1928 - accuracy: 0.9463
1177/1500 [======================>.......] - ETA: 2s - loss: 0.1925 - accuracy: 0.9464
1184/1500 [======================>.......] - ETA: 2s - loss: 0.1922 - accuracy: 0.9466
1191/1500 [======================>.......] - ETA: 2s - loss: 0.1923 - accuracy: 0.9465
1198/1500 [======================>.......] - ETA: 2s - loss: 0.1924 - accuracy: 0.9465
1205/1500 [=======================>......] - ETA: 2s - loss: 0.1929 - accuracy: 0.9464
1211/1500 [=======================>......] - ETA: 2s - loss: 0.1927 - accuracy: 0.9464
1218/1500 [=======================>......] - ETA: 2s - loss: 0.1929 - accuracy: 0.9464
1225/1500 [=======================>......] - ETA: 2s - loss: 0.1927 - accuracy: 0.9464
1231/1500 [=======================>......] - ETA: 2s - loss: 0.1928 - accuracy: 0.9464
1238/1500 [=======================>......] - ETA: 2s - loss: 0.1925 - accuracy: 0.9465
1244/1500 [=======================>......] - ETA: 2s - loss: 0.1922 - accuracy: 0.9466
1251/1500 [========================>.....] - ETA: 2s - loss: 0.1921 - accuracy: 0.9466
1258/1500 [========================>.....] - ETA: 1s - loss: 0.1917 - accuracy: 0.9467
1265/1500 [========================>.....] - ETA: 1s - loss: 0.1917 - accuracy: 0.9468
1272/1500 [========================>.....] - ETA: 1s - loss: 0.1919 - accuracy: 0.9466
1279/1500 [========================>.....] - ETA: 1s - loss: 0.1918 - accuracy: 0.9467
1286/1500 [========================>.....] - ETA: 1s - loss: 0.1915 - accuracy: 0.9468
1293/1500 [========================>.....] - ETA: 1s - loss: 0.1912 - accuracy: 0.9469
1300/1500 [=========================>....] - ETA: 1s - loss: 0.1913 - accuracy: 0.9469
1307/1500 [=========================>....] - ETA: 1s - loss: 0.1912 - accuracy: 0.9470
1314/1500 [=========================>....] - ETA: 1s - loss: 0.1912 - accuracy: 0.9470
1321/1500 [=========================>....] - ETA: 1s - loss: 0.1909 - accuracy: 0.9471
1328/1500 [=========================>....] - ETA: 1s - loss: 0.1910 - accuracy: 0.9469
1334/1500 [=========================>....] - ETA: 1s - loss: 0.1910 - accuracy: 0.9470
1341/1500 [=========================>....] - ETA: 1s - loss: 0.1907 - accuracy: 0.9470
1348/1500 [=========================>....] - ETA: 1s - loss: 0.1908 - accuracy: 0.9470
1354/1500 [==========================>...] - ETA: 1s - loss: 0.1910 - accuracy: 0.9469
1361/1500 [==========================>...] - ETA: 1s - loss: 0.1908 - accuracy: 0.9470
1368/1500 [==========================>...] - ETA: 1s - loss: 0.1905 - accuracy: 0.9471
1375/1500 [==========================>...] - ETA: 1s - loss: 0.1907 - accuracy: 0.9471
1382/1500 [==========================>...] - ETA: 0s - loss: 0.1907 - accuracy: 0.9470
1389/1500 [==========================>...] - ETA: 0s - loss: 0.1907 - accuracy: 0.9471
1395/1500 [==========================>...] - ETA: 0s - loss: 0.1904 - accuracy: 0.9472
1402/1500 [===========================>..] - ETA: 0s - loss: 0.1904 - accuracy: 0.9472
1409/1500 [===========================>..] - ETA: 0s - loss: 0.1907 - accuracy: 0.9472
1414/1500 [===========================>..] - ETA: 0s - loss: 0.1908 - accuracy: 0.9472
1419/1500 [===========================>..] - ETA: 0s - loss: 0.1909 - accuracy: 0.9472
1426/1500 [===========================>..] - ETA: 0s - loss: 0.1910 - accuracy: 0.9471
1433/1500 [===========================>..] - ETA: 0s - loss: 0.1910 - accuracy: 0.9471
1440/1500 [===========================>..] - ETA: 0s - loss: 0.1912 - accuracy: 0.9470
1447/1500 [===========================>..] - ETA: 0s - loss: 0.1916 - accuracy: 0.9469
1454/1500 [============================>.] - ETA: 0s - loss: 0.1914 - accuracy: 0.9470
1461/1500 [============================>.] - ETA: 0s - loss: 0.1913 - accuracy: 0.9470
1468/1500 [============================>.] - ETA: 0s - loss: 0.1913 - accuracy: 0.9470
1475/1500 [============================>.] - ETA: 0s - loss: 0.1913 - accuracy: 0.9470
1482/1500 [============================>.] - ETA: 0s - loss: 0.1913 - accuracy: 0.9469
1489/1500 [============================>.] - ETA: 0s - loss: 0.1911 - accuracy: 0.9470
1496/1500 [============================>.] - ETA: 0s - loss: 0.1910 - accuracy: 0.9471
1500/1500 [==============================] - 12s 8ms/step - loss: 0.1910 - accuracy: 0.9471

1500/1500 [==============================] - 15s 10ms/step - loss: 0.1910 - accuracy: 0.9471 - val_loss: 0.1925 - val_accuracy: 0.9467
Epoch 9/10

   1/1500 [..............................] - ETA: 28s - loss: 0.1502 - accuracy: 0.9688
   8/1500 [..............................] - ETA: 11s - loss: 0.1632 - accuracy: 0.9570
  14/1500 [..............................] - ETA: 12s - loss: 0.1852 - accuracy: 0.9509
  21/1500 [..............................] - ETA: 11s - loss: 0.1955 - accuracy: 0.9494
  26/1500 [..............................] - ETA: 12s - loss: 0.1944 - accuracy: 0.9507
  33/1500 [..............................] - ETA: 12s - loss: 0.1838 - accuracy: 0.9527
  40/1500 [..............................] - ETA: 12s - loss: 0.1774 - accuracy: 0.9547
  47/1500 [..............................] - ETA: 12s - loss: 0.1692 - accuracy: 0.9561
  54/1500 [>.............................] - ETA: 11s - loss: 0.1732 - accuracy: 0.9543
  61/1500 [>.............................] - ETA: 11s - loss: 0.1834 - accuracy: 0.9508
  67/1500 [>.............................] - ETA: 11s - loss: 0.1929 - accuracy: 0.9496
  74/1500 [>.............................] - ETA: 11s - loss: 0.1831 - accuracy: 0.9531
  81/1500 [>.............................] - ETA: 11s - loss: 0.1777 - accuracy: 0.9537
  86/1500 [>.............................] - ETA: 11s - loss: 0.1796 - accuracy: 0.9546
  93/1500 [>.............................] - ETA: 11s - loss: 0.1829 - accuracy: 0.9519
 100/1500 [=>............................] - ETA: 11s - loss: 0.1854 - accuracy: 0.9497
 108/1500 [=>............................] - ETA: 11s - loss: 0.1823 - accuracy: 0.9505
 116/1500 [=>............................] - ETA: 11s - loss: 0.1806 - accuracy: 0.9507
 123/1500 [=>............................] - ETA: 11s - loss: 0.1801 - accuracy: 0.9507
 130/1500 [=>............................] - ETA: 10s - loss: 0.1770 - accuracy: 0.9514
 136/1500 [=>............................] - ETA: 10s - loss: 0.1770 - accuracy: 0.9517
 143/1500 [=>............................] - ETA: 10s - loss: 0.1766 - accuracy: 0.9519
 150/1500 [==>...........................] - ETA: 10s - loss: 0.1772 - accuracy: 0.9517
 157/1500 [==>...........................] - ETA: 10s - loss: 0.1777 - accuracy: 0.9510
 164/1500 [==>...........................] - ETA: 10s - loss: 0.1773 - accuracy: 0.9512
 171/1500 [==>...........................] - ETA: 10s - loss: 0.1820 - accuracy: 0.9499
 178/1500 [==>...........................] - ETA: 10s - loss: 0.1823 - accuracy: 0.9498
 184/1500 [==>...........................] - ETA: 10s - loss: 0.1837 - accuracy: 0.9499
 191/1500 [==>...........................] - ETA: 10s - loss: 0.1837 - accuracy: 0.9499
 198/1500 [==>...........................] - ETA: 10s - loss: 0.1857 - accuracy: 0.9501
 205/1500 [===>..........................] - ETA: 10s - loss: 0.1873 - accuracy: 0.9495
 212/1500 [===>..........................] - ETA: 10s - loss: 0.1882 - accuracy: 0.9493
 219/1500 [===>..........................] - ETA: 10s - loss: 0.1899 - accuracy: 0.9481
 226/1500 [===>..........................] - ETA: 10s - loss: 0.1913 - accuracy: 0.9470
 233/1500 [===>..........................] - ETA: 10s - loss: 0.1918 - accuracy: 0.9469
 240/1500 [===>..........................] - ETA: 9s - loss: 0.1908 - accuracy: 0.9471 
 247/1500 [===>..........................] - ETA: 9s - loss: 0.1912 - accuracy: 0.9474
 254/1500 [====>.........................] - ETA: 9s - loss: 0.1909 - accuracy: 0.9475
 261/1500 [====>.........................] - ETA: 9s - loss: 0.1917 - accuracy: 0.9470
 268/1500 [====>.........................] - ETA: 9s - loss: 0.1902 - accuracy: 0.9474
 275/1500 [====>.........................] - ETA: 9s - loss: 0.1905 - accuracy: 0.9472
 282/1500 [====>.........................] - ETA: 9s - loss: 0.1895 - accuracy: 0.9477
 289/1500 [====>.........................] - ETA: 9s - loss: 0.1906 - accuracy: 0.9477
 296/1500 [====>.........................] - ETA: 9s - loss: 0.1893 - accuracy: 0.9482
 303/1500 [=====>........................] - ETA: 9s - loss: 0.1879 - accuracy: 0.9487
 310/1500 [=====>........................] - ETA: 9s - loss: 0.1876 - accuracy: 0.9492
 316/1500 [=====>........................] - ETA: 9s - loss: 0.1865 - accuracy: 0.9494
 323/1500 [=====>........................] - ETA: 9s - loss: 0.1848 - accuracy: 0.9500
 330/1500 [=====>........................] - ETA: 9s - loss: 0.1866 - accuracy: 0.9494
 337/1500 [=====>........................] - ETA: 9s - loss: 0.1856 - accuracy: 0.9497
 344/1500 [=====>........................] - ETA: 9s - loss: 0.1863 - accuracy: 0.9494
 351/1500 [======>.......................] - ETA: 9s - loss: 0.1850 - accuracy: 0.9499
 357/1500 [======>.......................] - ETA: 9s - loss: 0.1856 - accuracy: 0.9497
 364/1500 [======>.......................] - ETA: 8s - loss: 0.1865 - accuracy: 0.9492
 371/1500 [======>.......................] - ETA: 8s - loss: 0.1859 - accuracy: 0.9491
 378/1500 [======>.......................] - ETA: 8s - loss: 0.1855 - accuracy: 0.9492
 385/1500 [======>.......................] - ETA: 8s - loss: 0.1851 - accuracy: 0.9492
 392/1500 [======>.......................] - ETA: 8s - loss: 0.1851 - accuracy: 0.9491
 399/1500 [======>.......................] - ETA: 8s - loss: 0.1852 - accuracy: 0.9489
 406/1500 [=======>......................] - ETA: 8s - loss: 0.1846 - accuracy: 0.9490
 413/1500 [=======>......................] - ETA: 8s - loss: 0.1847 - accuracy: 0.9491
 420/1500 [=======>......................] - ETA: 8s - loss: 0.1850 - accuracy: 0.9490
 427/1500 [=======>......................] - ETA: 8s - loss: 0.1855 - accuracy: 0.9488
 433/1500 [=======>......................] - ETA: 8s - loss: 0.1851 - accuracy: 0.9488
 440/1500 [=======>......................] - ETA: 8s - loss: 0.1845 - accuracy: 0.9492
 447/1500 [=======>......................] - ETA: 8s - loss: 0.1838 - accuracy: 0.9493
 454/1500 [========>.....................] - ETA: 8s - loss: 0.1834 - accuracy: 0.9493
 461/1500 [========>.....................] - ETA: 8s - loss: 0.1827 - accuracy: 0.9494
 467/1500 [========>.....................] - ETA: 8s - loss: 0.1828 - accuracy: 0.9493
 474/1500 [========>.....................] - ETA: 8s - loss: 0.1823 - accuracy: 0.9496
 481/1500 [========>.....................] - ETA: 8s - loss: 0.1823 - accuracy: 0.9495
 488/1500 [========>.....................] - ETA: 7s - loss: 0.1827 - accuracy: 0.9492
 495/1500 [========>.....................] - ETA: 7s - loss: 0.1821 - accuracy: 0.9492
 502/1500 [=========>....................] - ETA: 7s - loss: 0.1819 - accuracy: 0.9493
 509/1500 [=========>....................] - ETA: 7s - loss: 0.1822 - accuracy: 0.9492
 516/1500 [=========>....................] - ETA: 7s - loss: 0.1829 - accuracy: 0.9492
 523/1500 [=========>....................] - ETA: 7s - loss: 0.1823 - accuracy: 0.9496
 530/1500 [=========>....................] - ETA: 7s - loss: 0.1814 - accuracy: 0.9499
 537/1500 [=========>....................] - ETA: 7s - loss: 0.1811 - accuracy: 0.9498
 544/1500 [=========>....................] - ETA: 7s - loss: 0.1809 - accuracy: 0.9496
 550/1500 [==========>...................] - ETA: 7s - loss: 0.1804 - accuracy: 0.9496
 556/1500 [==========>...................] - ETA: 7s - loss: 0.1805 - accuracy: 0.9494
 563/1500 [==========>...................] - ETA: 7s - loss: 0.1803 - accuracy: 0.9495
 570/1500 [==========>...................] - ETA: 7s - loss: 0.1808 - accuracy: 0.9495
 577/1500 [==========>...................] - ETA: 7s - loss: 0.1805 - accuracy: 0.9497
 584/1500 [==========>...................] - ETA: 7s - loss: 0.1803 - accuracy: 0.9497
 591/1500 [==========>...................] - ETA: 7s - loss: 0.1802 - accuracy: 0.9498
 598/1500 [==========>...................] - ETA: 7s - loss: 0.1802 - accuracy: 0.9496
 605/1500 [===========>..................] - ETA: 7s - loss: 0.1799 - accuracy: 0.9495
 612/1500 [===========>..................] - ETA: 7s - loss: 0.1792 - accuracy: 0.9499
 619/1500 [===========>..................] - ETA: 6s - loss: 0.1793 - accuracy: 0.9499
 626/1500 [===========>..................] - ETA: 6s - loss: 0.1801 - accuracy: 0.9497
 633/1500 [===========>..................] - ETA: 6s - loss: 0.1801 - accuracy: 0.9498
 640/1500 [===========>..................] - ETA: 6s - loss: 0.1805 - accuracy: 0.9497
 647/1500 [===========>..................] - ETA: 6s - loss: 0.1796 - accuracy: 0.9500
 654/1500 [============>.................] - ETA: 6s - loss: 0.1798 - accuracy: 0.9498
 661/1500 [============>.................] - ETA: 6s - loss: 0.1792 - accuracy: 0.9498
 667/1500 [============>.................] - ETA: 6s - loss: 0.1791 - accuracy: 0.9499
 674/1500 [============>.................] - ETA: 6s - loss: 0.1788 - accuracy: 0.9501
 681/1500 [============>.................] - ETA: 6s - loss: 0.1797 - accuracy: 0.9498
 688/1500 [============>.................] - ETA: 6s - loss: 0.1795 - accuracy: 0.9498
 695/1500 [============>.................] - ETA: 6s - loss: 0.1795 - accuracy: 0.9498
 702/1500 [=============>................] - ETA: 6s - loss: 0.1795 - accuracy: 0.9497
 709/1500 [=============>................] - ETA: 6s - loss: 0.1798 - accuracy: 0.9497
 716/1500 [=============>................] - ETA: 6s - loss: 0.1799 - accuracy: 0.9496
 723/1500 [=============>................] - ETA: 6s - loss: 0.1796 - accuracy: 0.9497
 730/1500 [=============>................] - ETA: 6s - loss: 0.1798 - accuracy: 0.9497
 737/1500 [=============>................] - ETA: 5s - loss: 0.1798 - accuracy: 0.9499
 744/1500 [=============>................] - ETA: 5s - loss: 0.1802 - accuracy: 0.9498
 751/1500 [==============>...............] - ETA: 5s - loss: 0.1800 - accuracy: 0.9499
 758/1500 [==============>...............] - ETA: 5s - loss: 0.1800 - accuracy: 0.9500
 765/1500 [==============>...............] - ETA: 5s - loss: 0.1799 - accuracy: 0.9502
 773/1500 [==============>...............] - ETA: 5s - loss: 0.1797 - accuracy: 0.9503
 780/1500 [==============>...............] - ETA: 5s - loss: 0.1793 - accuracy: 0.9504
 788/1500 [==============>...............] - ETA: 5s - loss: 0.1796 - accuracy: 0.9503
 796/1500 [==============>...............] - ETA: 5s - loss: 0.1794 - accuracy: 0.9506
 803/1500 [===============>..............] - ETA: 5s - loss: 0.1790 - accuracy: 0.9508
 810/1500 [===============>..............] - ETA: 5s - loss: 0.1794 - accuracy: 0.9506
 818/1500 [===============>..............] - ETA: 5s - loss: 0.1797 - accuracy: 0.9505
 826/1500 [===============>..............] - ETA: 5s - loss: 0.1796 - accuracy: 0.9504
 832/1500 [===============>..............] - ETA: 5s - loss: 0.1798 - accuracy: 0.9504
 839/1500 [===============>..............] - ETA: 5s - loss: 0.1797 - accuracy: 0.9504
 846/1500 [===============>..............] - ETA: 5s - loss: 0.1793 - accuracy: 0.9505
 852/1500 [================>.............] - ETA: 5s - loss: 0.1797 - accuracy: 0.9503
 859/1500 [================>.............] - ETA: 4s - loss: 0.1797 - accuracy: 0.9502
 866/1500 [================>.............] - ETA: 4s - loss: 0.1795 - accuracy: 0.9502
 873/1500 [================>.............] - ETA: 4s - loss: 0.1797 - accuracy: 0.9501
 879/1500 [================>.............] - ETA: 4s - loss: 0.1798 - accuracy: 0.9500
 885/1500 [================>.............] - ETA: 4s - loss: 0.1799 - accuracy: 0.9501
 891/1500 [================>.............] - ETA: 4s - loss: 0.1798 - accuracy: 0.9500
 898/1500 [================>.............] - ETA: 4s - loss: 0.1796 - accuracy: 0.9500
 905/1500 [=================>............] - ETA: 4s - loss: 0.1797 - accuracy: 0.9500
 912/1500 [=================>............] - ETA: 4s - loss: 0.1797 - accuracy: 0.9501
 919/1500 [=================>............] - ETA: 4s - loss: 0.1800 - accuracy: 0.9500
 926/1500 [=================>............] - ETA: 4s - loss: 0.1802 - accuracy: 0.9500
 933/1500 [=================>............] - ETA: 4s - loss: 0.1802 - accuracy: 0.9500
 940/1500 [=================>............] - ETA: 4s - loss: 0.1799 - accuracy: 0.9501
 946/1500 [=================>............] - ETA: 4s - loss: 0.1799 - accuracy: 0.9501
 953/1500 [==================>...........] - ETA: 4s - loss: 0.1801 - accuracy: 0.9501
 960/1500 [==================>...........] - ETA: 4s - loss: 0.1806 - accuracy: 0.9500
 967/1500 [==================>...........] - ETA: 4s - loss: 0.1816 - accuracy: 0.9499
 974/1500 [==================>...........] - ETA: 4s - loss: 0.1818 - accuracy: 0.9499
 981/1500 [==================>...........] - ETA: 4s - loss: 0.1815 - accuracy: 0.9500
 988/1500 [==================>...........] - ETA: 4s - loss: 0.1820 - accuracy: 0.9499
 994/1500 [==================>...........] - ETA: 3s - loss: 0.1818 - accuracy: 0.9499
1000/1500 [===================>..........] - ETA: 3s - loss: 0.1814 - accuracy: 0.9501
1007/1500 [===================>..........] - ETA: 3s - loss: 0.1817 - accuracy: 0.9498
1013/1500 [===================>..........] - ETA: 3s - loss: 0.1813 - accuracy: 0.9500
1020/1500 [===================>..........] - ETA: 3s - loss: 0.1811 - accuracy: 0.9500
1026/1500 [===================>..........] - ETA: 3s - loss: 0.1810 - accuracy: 0.9500
1033/1500 [===================>..........] - ETA: 3s - loss: 0.1812 - accuracy: 0.9499
1040/1500 [===================>..........] - ETA: 3s - loss: 0.1811 - accuracy: 0.9499
1047/1500 [===================>..........] - ETA: 3s - loss: 0.1807 - accuracy: 0.9501
1054/1500 [====================>.........] - ETA: 3s - loss: 0.1804 - accuracy: 0.9502
1061/1500 [====================>.........] - ETA: 3s - loss: 0.1803 - accuracy: 0.9502
1068/1500 [====================>.........] - ETA: 3s - loss: 0.1799 - accuracy: 0.9503
1075/1500 [====================>.........] - ETA: 3s - loss: 0.1799 - accuracy: 0.9503
1082/1500 [====================>.........] - ETA: 3s - loss: 0.1802 - accuracy: 0.9501
1089/1500 [====================>.........] - ETA: 3s - loss: 0.1804 - accuracy: 0.9501
1096/1500 [====================>.........] - ETA: 3s - loss: 0.1802 - accuracy: 0.9501
1102/1500 [=====================>........] - ETA: 3s - loss: 0.1800 - accuracy: 0.9502
1108/1500 [=====================>........] - ETA: 3s - loss: 0.1799 - accuracy: 0.9502
1115/1500 [=====================>........] - ETA: 3s - loss: 0.1799 - accuracy: 0.9502
1122/1500 [=====================>........] - ETA: 2s - loss: 0.1798 - accuracy: 0.9502
1129/1500 [=====================>........] - ETA: 2s - loss: 0.1802 - accuracy: 0.9502
1134/1500 [=====================>........] - ETA: 2s - loss: 0.1800 - accuracy: 0.9503
1140/1500 [=====================>........] - ETA: 2s - loss: 0.1802 - accuracy: 0.9503
1147/1500 [=====================>........] - ETA: 2s - loss: 0.1803 - accuracy: 0.9501
1154/1500 [======================>.......] - ETA: 2s - loss: 0.1799 - accuracy: 0.9503
1161/1500 [======================>.......] - ETA: 2s - loss: 0.1801 - accuracy: 0.9502
1167/1500 [======================>.......] - ETA: 2s - loss: 0.1800 - accuracy: 0.9502
1174/1500 [======================>.......] - ETA: 2s - loss: 0.1799 - accuracy: 0.9501
1181/1500 [======================>.......] - ETA: 2s - loss: 0.1803 - accuracy: 0.9501
1187/1500 [======================>.......] - ETA: 2s - loss: 0.1803 - accuracy: 0.9501
1189/1500 [======================>.......] - ETA: 2s - loss: 0.1803 - accuracy: 0.9501
1196/1500 [======================>.......] - ETA: 2s - loss: 0.1803 - accuracy: 0.9501
1203/1500 [=======================>......] - ETA: 2s - loss: 0.1802 - accuracy: 0.9502
1210/1500 [=======================>......] - ETA: 2s - loss: 0.1805 - accuracy: 0.9502
1217/1500 [=======================>......] - ETA: 2s - loss: 0.1807 - accuracy: 0.9501
1222/1500 [=======================>......] - ETA: 2s - loss: 0.1805 - accuracy: 0.9501
1228/1500 [=======================>......] - ETA: 2s - loss: 0.1805 - accuracy: 0.9501
1235/1500 [=======================>......] - ETA: 2s - loss: 0.1806 - accuracy: 0.9501
1243/1500 [=======================>......] - ETA: 2s - loss: 0.1807 - accuracy: 0.9500
1250/1500 [========================>.....] - ETA: 1s - loss: 0.1808 - accuracy: 0.9501
1257/1500 [========================>.....] - ETA: 1s - loss: 0.1812 - accuracy: 0.9499
1264/1500 [========================>.....] - ETA: 1s - loss: 0.1812 - accuracy: 0.9498
1271/1500 [========================>.....] - ETA: 1s - loss: 0.1813 - accuracy: 0.9498
1277/1500 [========================>.....] - ETA: 1s - loss: 0.1812 - accuracy: 0.9498
1283/1500 [========================>.....] - ETA: 1s - loss: 0.1812 - accuracy: 0.9498
1289/1500 [========================>.....] - ETA: 1s - loss: 0.1810 - accuracy: 0.9498
1295/1500 [========================>.....] - ETA: 1s - loss: 0.1808 - accuracy: 0.9500
1302/1500 [=========================>....] - ETA: 1s - loss: 0.1811 - accuracy: 0.9498
1309/1500 [=========================>....] - ETA: 1s - loss: 0.1808 - accuracy: 0.9499
1313/1500 [=========================>....] - ETA: 1s - loss: 0.1807 - accuracy: 0.9499
1320/1500 [=========================>....] - ETA: 1s - loss: 0.1807 - accuracy: 0.9498
1328/1500 [=========================>....] - ETA: 1s - loss: 0.1808 - accuracy: 0.9498
1335/1500 [=========================>....] - ETA: 1s - loss: 0.1805 - accuracy: 0.9499
1341/1500 [=========================>....] - ETA: 1s - loss: 0.1805 - accuracy: 0.9499
1349/1500 [=========================>....] - ETA: 1s - loss: 0.1803 - accuracy: 0.9499
1356/1500 [==========================>...] - ETA: 1s - loss: 0.1800 - accuracy: 0.9500
1363/1500 [==========================>...] - ETA: 1s - loss: 0.1798 - accuracy: 0.9501
1370/1500 [==========================>...] - ETA: 1s - loss: 0.1801 - accuracy: 0.9501
1377/1500 [==========================>...] - ETA: 0s - loss: 0.1798 - accuracy: 0.9501
1383/1500 [==========================>...] - ETA: 0s - loss: 0.1801 - accuracy: 0.9500
1390/1500 [==========================>...] - ETA: 0s - loss: 0.1801 - accuracy: 0.9500
1397/1500 [==========================>...] - ETA: 0s - loss: 0.1800 - accuracy: 0.9500
1404/1500 [===========================>..] - ETA: 0s - loss: 0.1799 - accuracy: 0.9500
1411/1500 [===========================>..] - ETA: 0s - loss: 0.1797 - accuracy: 0.9501
1418/1500 [===========================>..] - ETA: 0s - loss: 0.1795 - accuracy: 0.9501
1425/1500 [===========================>..] - ETA: 0s - loss: 0.1796 - accuracy: 0.9500
1431/1500 [===========================>..] - ETA: 0s - loss: 0.1793 - accuracy: 0.9501
1437/1500 [===========================>..] - ETA: 0s - loss: 0.1795 - accuracy: 0.9500
1444/1500 [===========================>..] - ETA: 0s - loss: 0.1794 - accuracy: 0.9500
1451/1500 [============================>.] - ETA: 0s - loss: 0.1796 - accuracy: 0.9499
1458/1500 [============================>.] - ETA: 0s - loss: 0.1798 - accuracy: 0.9499
1465/1500 [============================>.] - ETA: 0s - loss: 0.1795 - accuracy: 0.9500
1472/1500 [============================>.] - ETA: 0s - loss: 0.1795 - accuracy: 0.9500
1479/1500 [============================>.] - ETA: 0s - loss: 0.1793 - accuracy: 0.9501
1486/1500 [============================>.] - ETA: 0s - loss: 0.1795 - accuracy: 0.9501
1493/1500 [============================>.] - ETA: 0s - loss: 0.1792 - accuracy: 0.9501
1500/1500 [==============================] - 12s 8ms/step - loss: 0.1790 - accuracy: 0.9501

1500/1500 [==============================] - 15s 10ms/step - loss: 0.1790 - accuracy: 0.9501 - val_loss: 0.1834 - val_accuracy: 0.9488
Epoch 10/10

   1/1500 [..............................] - ETA: 39s - loss: 0.1990 - accuracy: 0.9062
   6/1500 [..............................] - ETA: 15s - loss: 0.1763 - accuracy: 0.9427
  12/1500 [..............................] - ETA: 14s - loss: 0.1974 - accuracy: 0.9349
  17/1500 [..............................] - ETA: 14s - loss: 0.1828 - accuracy: 0.9412
  23/1500 [..............................] - ETA: 14s - loss: 0.1686 - accuracy: 0.9484
  29/1500 [..............................] - ETA: 14s - loss: 0.1794 - accuracy: 0.9461
  35/1500 [..............................] - ETA: 13s - loss: 0.1835 - accuracy: 0.9473
  42/1500 [..............................] - ETA: 13s - loss: 0.1766 - accuracy: 0.9501
  48/1500 [..............................] - ETA: 13s - loss: 0.1783 - accuracy: 0.9492
  56/1500 [>.............................] - ETA: 12s - loss: 0.1721 - accuracy: 0.9503
  63/1500 [>.............................] - ETA: 12s - loss: 0.1710 - accuracy: 0.9514
  70/1500 [>.............................] - ETA: 12s - loss: 0.1731 - accuracy: 0.9478
  77/1500 [>.............................] - ETA: 12s - loss: 0.1713 - accuracy: 0.9493
  84/1500 [>.............................] - ETA: 12s - loss: 0.1693 - accuracy: 0.9490
  89/1500 [>.............................] - ETA: 12s - loss: 0.1672 - accuracy: 0.9494
  94/1500 [>.............................] - ETA: 12s - loss: 0.1657 - accuracy: 0.9501
 100/1500 [=>............................] - ETA: 12s - loss: 0.1668 - accuracy: 0.9506
 106/1500 [=>............................] - ETA: 12s - loss: 0.1702 - accuracy: 0.9502
 111/1500 [=>............................] - ETA: 12s - loss: 0.1707 - accuracy: 0.9505
 117/1500 [=>............................] - ETA: 12s - loss: 0.1703 - accuracy: 0.9503
 123/1500 [=>............................] - ETA: 12s - loss: 0.1696 - accuracy: 0.9512
 129/1500 [=>............................] - ETA: 12s - loss: 0.1718 - accuracy: 0.9506
 134/1500 [=>............................] - ETA: 12s - loss: 0.1753 - accuracy: 0.9496
 139/1500 [=>............................] - ETA: 12s - loss: 0.1754 - accuracy: 0.9494
 143/1500 [=>............................] - ETA: 12s - loss: 0.1748 - accuracy: 0.9502
 148/1500 [=>............................] - ETA: 12s - loss: 0.1760 - accuracy: 0.9506
 154/1500 [==>...........................] - ETA: 12s - loss: 0.1773 - accuracy: 0.9499
 160/1500 [==>...........................] - ETA: 12s - loss: 0.1763 - accuracy: 0.9498
 166/1500 [==>...........................] - ETA: 12s - loss: 0.1769 - accuracy: 0.9501
 173/1500 [==>...........................] - ETA: 12s - loss: 0.1770 - accuracy: 0.9503
 180/1500 [==>...........................] - ETA: 12s - loss: 0.1782 - accuracy: 0.9500
 187/1500 [==>...........................] - ETA: 12s - loss: 0.1792 - accuracy: 0.9495
 193/1500 [==>...........................] - ETA: 11s - loss: 0.1780 - accuracy: 0.9496
 200/1500 [===>..........................] - ETA: 11s - loss: 0.1795 - accuracy: 0.9497
 205/1500 [===>..........................] - ETA: 11s - loss: 0.1786 - accuracy: 0.9497
 212/1500 [===>..........................] - ETA: 11s - loss: 0.1766 - accuracy: 0.9503
 219/1500 [===>..........................] - ETA: 11s - loss: 0.1760 - accuracy: 0.9505
 225/1500 [===>..........................] - ETA: 11s - loss: 0.1757 - accuracy: 0.9506
 231/1500 [===>..........................] - ETA: 11s - loss: 0.1743 - accuracy: 0.9506
 237/1500 [===>..........................] - ETA: 11s - loss: 0.1739 - accuracy: 0.9504
 242/1500 [===>..........................] - ETA: 11s - loss: 0.1739 - accuracy: 0.9502
 247/1500 [===>..........................] - ETA: 11s - loss: 0.1734 - accuracy: 0.9503
 253/1500 [====>.........................] - ETA: 11s - loss: 0.1728 - accuracy: 0.9503
 258/1500 [====>.........................] - ETA: 11s - loss: 0.1725 - accuracy: 0.9503
 265/1500 [====>.........................] - ETA: 11s - loss: 0.1733 - accuracy: 0.9500
 272/1500 [====>.........................] - ETA: 11s - loss: 0.1734 - accuracy: 0.9500
 279/1500 [====>.........................] - ETA: 11s - loss: 0.1727 - accuracy: 0.9503
 285/1500 [====>.........................] - ETA: 11s - loss: 0.1731 - accuracy: 0.9502
 288/1500 [====>.........................] - ETA: 11s - loss: 0.1720 - accuracy: 0.9505
 292/1500 [====>.........................] - ETA: 11s - loss: 0.1716 - accuracy: 0.9508
 297/1500 [====>.........................] - ETA: 11s - loss: 0.1710 - accuracy: 0.9511
 303/1500 [=====>........................] - ETA: 11s - loss: 0.1704 - accuracy: 0.9514
 308/1500 [=====>........................] - ETA: 11s - loss: 0.1709 - accuracy: 0.9513
 313/1500 [=====>........................] - ETA: 11s - loss: 0.1709 - accuracy: 0.9513
 319/1500 [=====>........................] - ETA: 11s - loss: 0.1708 - accuracy: 0.9513
 326/1500 [=====>........................] - ETA: 10s - loss: 0.1729 - accuracy: 0.9508
 333/1500 [=====>........................] - ETA: 10s - loss: 0.1739 - accuracy: 0.9505
 340/1500 [=====>........................] - ETA: 10s - loss: 0.1732 - accuracy: 0.9507
 345/1500 [=====>........................] - ETA: 10s - loss: 0.1733 - accuracy: 0.9506
 351/1500 [======>.......................] - ETA: 10s - loss: 0.1733 - accuracy: 0.9508
 358/1500 [======>.......................] - ETA: 10s - loss: 0.1721 - accuracy: 0.9510
 364/1500 [======>.......................] - ETA: 10s - loss: 0.1722 - accuracy: 0.9508
 369/1500 [======>.......................] - ETA: 10s - loss: 0.1718 - accuracy: 0.9509
 375/1500 [======>.......................] - ETA: 10s - loss: 0.1708 - accuracy: 0.9513
 381/1500 [======>.......................] - ETA: 10s - loss: 0.1696 - accuracy: 0.9519
 388/1500 [======>.......................] - ETA: 10s - loss: 0.1707 - accuracy: 0.9516
 395/1500 [======>.......................] - ETA: 10s - loss: 0.1703 - accuracy: 0.9518
 402/1500 [=======>......................] - ETA: 10s - loss: 0.1700 - accuracy: 0.9520
 409/1500 [=======>......................] - ETA: 9s - loss: 0.1692 - accuracy: 0.9524 
 416/1500 [=======>......................] - ETA: 9s - loss: 0.1683 - accuracy: 0.9527
 423/1500 [=======>......................] - ETA: 9s - loss: 0.1674 - accuracy: 0.9530
 430/1500 [=======>......................] - ETA: 9s - loss: 0.1676 - accuracy: 0.9531
 437/1500 [=======>......................] - ETA: 9s - loss: 0.1672 - accuracy: 0.9534
 444/1500 [=======>......................] - ETA: 9s - loss: 0.1671 - accuracy: 0.9533
 450/1500 [========>.....................] - ETA: 9s - loss: 0.1668 - accuracy: 0.9535
 457/1500 [========>.....................] - ETA: 9s - loss: 0.1678 - accuracy: 0.9534
 464/1500 [========>.....................] - ETA: 9s - loss: 0.1675 - accuracy: 0.9535
 471/1500 [========>.....................] - ETA: 9s - loss: 0.1671 - accuracy: 0.9537
 478/1500 [========>.....................] - ETA: 9s - loss: 0.1667 - accuracy: 0.9538
 485/1500 [========>.....................] - ETA: 9s - loss: 0.1665 - accuracy: 0.9541
 492/1500 [========>.....................] - ETA: 9s - loss: 0.1659 - accuracy: 0.9543
 499/1500 [========>.....................] - ETA: 8s - loss: 0.1664 - accuracy: 0.9543
 506/1500 [=========>....................] - ETA: 8s - loss: 0.1673 - accuracy: 0.9542
 513/1500 [=========>....................] - ETA: 8s - loss: 0.1682 - accuracy: 0.9540
 520/1500 [=========>....................] - ETA: 8s - loss: 0.1677 - accuracy: 0.9541
 527/1500 [=========>....................] - ETA: 8s - loss: 0.1681 - accuracy: 0.9540
 534/1500 [=========>....................] - ETA: 8s - loss: 0.1687 - accuracy: 0.9540
 541/1500 [=========>....................] - ETA: 8s - loss: 0.1689 - accuracy: 0.9538
 548/1500 [=========>....................] - ETA: 8s - loss: 0.1678 - accuracy: 0.9541
 554/1500 [==========>...................] - ETA: 8s - loss: 0.1678 - accuracy: 0.9541
 561/1500 [==========>...................] - ETA: 8s - loss: 0.1672 - accuracy: 0.9543
 568/1500 [==========>...................] - ETA: 8s - loss: 0.1674 - accuracy: 0.9542
 575/1500 [==========>...................] - ETA: 8s - loss: 0.1676 - accuracy: 0.9542
 583/1500 [==========>...................] - ETA: 8s - loss: 0.1672 - accuracy: 0.9542
 590/1500 [==========>...................] - ETA: 7s - loss: 0.1680 - accuracy: 0.9541
 596/1500 [==========>...................] - ETA: 7s - loss: 0.1678 - accuracy: 0.9543
 602/1500 [===========>..................] - ETA: 7s - loss: 0.1686 - accuracy: 0.9542
 610/1500 [===========>..................] - ETA: 7s - loss: 0.1679 - accuracy: 0.9544
 616/1500 [===========>..................] - ETA: 7s - loss: 0.1681 - accuracy: 0.9543
 622/1500 [===========>..................] - ETA: 7s - loss: 0.1679 - accuracy: 0.9544
 629/1500 [===========>..................] - ETA: 7s - loss: 0.1682 - accuracy: 0.9543
 636/1500 [===========>..................] - ETA: 7s - loss: 0.1682 - accuracy: 0.9541
 643/1500 [===========>..................] - ETA: 7s - loss: 0.1682 - accuracy: 0.9540
 649/1500 [===========>..................] - ETA: 7s - loss: 0.1681 - accuracy: 0.9539
 656/1500 [============>.................] - ETA: 7s - loss: 0.1680 - accuracy: 0.9539
 663/1500 [============>.................] - ETA: 7s - loss: 0.1677 - accuracy: 0.9540
 670/1500 [============>.................] - ETA: 7s - loss: 0.1680 - accuracy: 0.9540
 676/1500 [============>.................] - ETA: 7s - loss: 0.1675 - accuracy: 0.9542
 683/1500 [============>.................] - ETA: 7s - loss: 0.1674 - accuracy: 0.9544
 690/1500 [============>.................] - ETA: 7s - loss: 0.1671 - accuracy: 0.9546
 697/1500 [============>.................] - ETA: 6s - loss: 0.1676 - accuracy: 0.9543
 703/1500 [=============>................] - ETA: 6s - loss: 0.1673 - accuracy: 0.9543
 710/1500 [=============>................] - ETA: 6s - loss: 0.1678 - accuracy: 0.9541
 716/1500 [=============>................] - ETA: 6s - loss: 0.1674 - accuracy: 0.9543
 721/1500 [=============>................] - ETA: 6s - loss: 0.1672 - accuracy: 0.9544
 728/1500 [=============>................] - ETA: 6s - loss: 0.1672 - accuracy: 0.9544
 735/1500 [=============>................] - ETA: 6s - loss: 0.1670 - accuracy: 0.9545
 741/1500 [=============>................] - ETA: 6s - loss: 0.1675 - accuracy: 0.9543
 748/1500 [=============>................] - ETA: 6s - loss: 0.1679 - accuracy: 0.9543
 755/1500 [==============>...............] - ETA: 6s - loss: 0.1681 - accuracy: 0.9543
 762/1500 [==============>...............] - ETA: 6s - loss: 0.1679 - accuracy: 0.9544
 769/1500 [==============>...............] - ETA: 6s - loss: 0.1682 - accuracy: 0.9542
 776/1500 [==============>...............] - ETA: 6s - loss: 0.1682 - accuracy: 0.9540
 780/1500 [==============>...............] - ETA: 6s - loss: 0.1681 - accuracy: 0.9540
 787/1500 [==============>...............] - ETA: 6s - loss: 0.1678 - accuracy: 0.9540
 794/1500 [==============>...............] - ETA: 6s - loss: 0.1675 - accuracy: 0.9541
 801/1500 [===============>..............] - ETA: 6s - loss: 0.1680 - accuracy: 0.9540
 808/1500 [===============>..............] - ETA: 5s - loss: 0.1679 - accuracy: 0.9541
 815/1500 [===============>..............] - ETA: 5s - loss: 0.1679 - accuracy: 0.9541
 822/1500 [===============>..............] - ETA: 5s - loss: 0.1673 - accuracy: 0.9544
 828/1500 [===============>..............] - ETA: 5s - loss: 0.1671 - accuracy: 0.9544
 830/1500 [===============>..............] - ETA: 5s - loss: 0.1671 - accuracy: 0.9544
 837/1500 [===============>..............] - ETA: 5s - loss: 0.1669 - accuracy: 0.9544
 844/1500 [===============>..............] - ETA: 5s - loss: 0.1669 - accuracy: 0.9543
 851/1500 [================>.............] - ETA: 5s - loss: 0.1671 - accuracy: 0.9541
 859/1500 [================>.............] - ETA: 5s - loss: 0.1668 - accuracy: 0.9542
 866/1500 [================>.............] - ETA: 5s - loss: 0.1668 - accuracy: 0.9541
 870/1500 [================>.............] - ETA: 5s - loss: 0.1664 - accuracy: 0.9543
 876/1500 [================>.............] - ETA: 5s - loss: 0.1664 - accuracy: 0.9542
 883/1500 [================>.............] - ETA: 5s - loss: 0.1661 - accuracy: 0.9543
 890/1500 [================>.............] - ETA: 5s - loss: 0.1665 - accuracy: 0.9541
 897/1500 [================>.............] - ETA: 5s - loss: 0.1667 - accuracy: 0.9542
 904/1500 [=================>............] - ETA: 5s - loss: 0.1669 - accuracy: 0.9541
 911/1500 [=================>............] - ETA: 5s - loss: 0.1665 - accuracy: 0.9541
 918/1500 [=================>............] - ETA: 5s - loss: 0.1664 - accuracy: 0.9541
 921/1500 [=================>............] - ETA: 5s - loss: 0.1663 - accuracy: 0.9541
 927/1500 [=================>............] - ETA: 4s - loss: 0.1663 - accuracy: 0.9542
 934/1500 [=================>............] - ETA: 4s - loss: 0.1670 - accuracy: 0.9540
 940/1500 [=================>............] - ETA: 4s - loss: 0.1669 - accuracy: 0.9540
 946/1500 [=================>............] - ETA: 4s - loss: 0.1667 - accuracy: 0.9540
 953/1500 [==================>...........] - ETA: 4s - loss: 0.1666 - accuracy: 0.9541
 960/1500 [==================>...........] - ETA: 4s - loss: 0.1664 - accuracy: 0.9542
 967/1500 [==================>...........] - ETA: 4s - loss: 0.1663 - accuracy: 0.9542
 974/1500 [==================>...........] - ETA: 4s - loss: 0.1664 - accuracy: 0.9541
 981/1500 [==================>...........] - ETA: 4s - loss: 0.1668 - accuracy: 0.9541
 988/1500 [==================>...........] - ETA: 4s - loss: 0.1665 - accuracy: 0.9542
 995/1500 [==================>...........] - ETA: 4s - loss: 0.1662 - accuracy: 0.9543
1002/1500 [===================>..........] - ETA: 4s - loss: 0.1661 - accuracy: 0.9544
1009/1500 [===================>..........] - ETA: 4s - loss: 0.1660 - accuracy: 0.9544
1016/1500 [===================>..........] - ETA: 4s - loss: 0.1659 - accuracy: 0.9545
1023/1500 [===================>..........] - ETA: 4s - loss: 0.1660 - accuracy: 0.9545
1030/1500 [===================>..........] - ETA: 4s - loss: 0.1660 - accuracy: 0.9544
1037/1500 [===================>..........] - ETA: 3s - loss: 0.1656 - accuracy: 0.9544
1044/1500 [===================>..........] - ETA: 3s - loss: 0.1656 - accuracy: 0.9543
1051/1500 [====================>.........] - ETA: 3s - loss: 0.1653 - accuracy: 0.9544
1058/1500 [====================>.........] - ETA: 3s - loss: 0.1651 - accuracy: 0.9544
1065/1500 [====================>.........] - ETA: 3s - loss: 0.1655 - accuracy: 0.9544
1072/1500 [====================>.........] - ETA: 3s - loss: 0.1656 - accuracy: 0.9543
1080/1500 [====================>.........] - ETA: 3s - loss: 0.1659 - accuracy: 0.9542
1087/1500 [====================>.........] - ETA: 3s - loss: 0.1659 - accuracy: 0.9541
1093/1500 [====================>.........] - ETA: 3s - loss: 0.1659 - accuracy: 0.9542
1100/1500 [=====================>........] - ETA: 3s - loss: 0.1660 - accuracy: 0.9542
1107/1500 [=====================>........] - ETA: 3s - loss: 0.1659 - accuracy: 0.9542
1113/1500 [=====================>........] - ETA: 3s - loss: 0.1658 - accuracy: 0.9542
1120/1500 [=====================>........] - ETA: 3s - loss: 0.1660 - accuracy: 0.9541
1127/1500 [=====================>........] - ETA: 3s - loss: 0.1657 - accuracy: 0.9542
1134/1500 [=====================>........] - ETA: 3s - loss: 0.1658 - accuracy: 0.9541
1141/1500 [=====================>........] - ETA: 3s - loss: 0.1661 - accuracy: 0.9540
1148/1500 [=====================>........] - ETA: 2s - loss: 0.1660 - accuracy: 0.9540
1155/1500 [======================>.......] - ETA: 2s - loss: 0.1661 - accuracy: 0.9540
1162/1500 [======================>.......] - ETA: 2s - loss: 0.1665 - accuracy: 0.9538
1169/1500 [======================>.......] - ETA: 2s - loss: 0.1666 - accuracy: 0.9537
1176/1500 [======================>.......] - ETA: 2s - loss: 0.1668 - accuracy: 0.9538
1183/1500 [======================>.......] - ETA: 2s - loss: 0.1668 - accuracy: 0.9539
1190/1500 [======================>.......] - ETA: 2s - loss: 0.1669 - accuracy: 0.9538
1197/1500 [======================>.......] - ETA: 2s - loss: 0.1671 - accuracy: 0.9538
1204/1500 [=======================>......] - ETA: 2s - loss: 0.1670 - accuracy: 0.9538
1211/1500 [=======================>......] - ETA: 2s - loss: 0.1672 - accuracy: 0.9537
1218/1500 [=======================>......] - ETA: 2s - loss: 0.1674 - accuracy: 0.9537
1225/1500 [=======================>......] - ETA: 2s - loss: 0.1673 - accuracy: 0.9537
1232/1500 [=======================>......] - ETA: 2s - loss: 0.1678 - accuracy: 0.9535
1239/1500 [=======================>......] - ETA: 2s - loss: 0.1677 - accuracy: 0.9535
1246/1500 [=======================>......] - ETA: 2s - loss: 0.1677 - accuracy: 0.9535
1253/1500 [========================>.....] - ETA: 2s - loss: 0.1677 - accuracy: 0.9536
1260/1500 [========================>.....] - ETA: 2s - loss: 0.1677 - accuracy: 0.9536
1267/1500 [========================>.....] - ETA: 1s - loss: 0.1676 - accuracy: 0.9537
1274/1500 [========================>.....] - ETA: 1s - loss: 0.1675 - accuracy: 0.9537
1281/1500 [========================>.....] - ETA: 1s - loss: 0.1673 - accuracy: 0.9538
1288/1500 [========================>.....] - ETA: 1s - loss: 0.1675 - accuracy: 0.9537
1295/1500 [========================>.....] - ETA: 1s - loss: 0.1675 - accuracy: 0.9536
1302/1500 [=========================>....] - ETA: 1s - loss: 0.1680 - accuracy: 0.9535
1309/1500 [=========================>....] - ETA: 1s - loss: 0.1681 - accuracy: 0.9534
1316/1500 [=========================>....] - ETA: 1s - loss: 0.1686 - accuracy: 0.9532
1323/1500 [=========================>....] - ETA: 1s - loss: 0.1685 - accuracy: 0.9532
1330/1500 [=========================>....] - ETA: 1s - loss: 0.1686 - accuracy: 0.9532
1337/1500 [=========================>....] - ETA: 1s - loss: 0.1688 - accuracy: 0.9532
1343/1500 [=========================>....] - ETA: 1s - loss: 0.1688 - accuracy: 0.9532
1350/1500 [==========================>...] - ETA: 1s - loss: 0.1687 - accuracy: 0.9533
1357/1500 [==========================>...] - ETA: 1s - loss: 0.1689 - accuracy: 0.9532
1364/1500 [==========================>...] - ETA: 1s - loss: 0.1694 - accuracy: 0.9531
1371/1500 [==========================>...] - ETA: 1s - loss: 0.1695 - accuracy: 0.9531
1378/1500 [==========================>...] - ETA: 1s - loss: 0.1695 - accuracy: 0.9531
1385/1500 [==========================>...] - ETA: 0s - loss: 0.1694 - accuracy: 0.9531
1392/1500 [==========================>...] - ETA: 0s - loss: 0.1697 - accuracy: 0.9531
1399/1500 [==========================>...] - ETA: 0s - loss: 0.1696 - accuracy: 0.9531
1406/1500 [===========================>..] - ETA: 0s - loss: 0.1700 - accuracy: 0.9530
1413/1500 [===========================>..] - ETA: 0s - loss: 0.1701 - accuracy: 0.9530
1420/1500 [===========================>..] - ETA: 0s - loss: 0.1701 - accuracy: 0.9530
1427/1500 [===========================>..] - ETA: 0s - loss: 0.1699 - accuracy: 0.9531
1434/1500 [===========================>..] - ETA: 0s - loss: 0.1696 - accuracy: 0.9532
1441/1500 [===========================>..] - ETA: 0s - loss: 0.1692 - accuracy: 0.9533
1448/1500 [===========================>..] - ETA: 0s - loss: 0.1690 - accuracy: 0.9534
1456/1500 [============================>.] - ETA: 0s - loss: 0.1689 - accuracy: 0.9534
1463/1500 [============================>.] - ETA: 0s - loss: 0.1689 - accuracy: 0.9534
1470/1500 [============================>.] - ETA: 0s - loss: 0.1690 - accuracy: 0.9534
1477/1500 [============================>.] - ETA: 0s - loss: 0.1687 - accuracy: 0.9535
1484/1500 [============================>.] - ETA: 0s - loss: 0.1684 - accuracy: 0.9535
1491/1500 [============================>.] - ETA: 0s - loss: 0.1686 - accuracy: 0.9535
1498/1500 [============================>.] - ETA: 0s - loss: 0.1685 - accuracy: 0.9535
1500/1500 [==============================] - 13s 8ms/step - loss: 0.1685 - accuracy: 0.9535

1500/1500 [==============================] - 15s 10ms/step - loss: 0.1685 - accuracy: 0.9535 - val_loss: 0.1750 - val_accuracy: 0.9502

history

plot(history)

Your turn! (3min)

Now retrain the same model but try with batch sizes of 16, 128, 512 while leaving everything else the same. How do the results (loss, accuracy, and compute time) compare?

# define model architecture
model <- keras_model_sequential() %>%
  layer_dense(units = 512, activation = 'relu', input_shape = n_feat) %>%
  layer_dense(units = 10, activation = 'softmax')

# define how the model is going to learn
model %>% compile(
  loss = "categorical_crossentropy",
  optimizer = "sgd",
  metrics = "accuracy"
  )

history <- model %>% fit(
  train_images, train_labels,
  validation_split = 0.2,
  batch_size = 128
  )

Epoch 1/10

2022-08-04 14:38:04.864648: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:113] Plugin optimizer for device_type GPU is enabled.

  1/375 [..............................] - ETA: 1:49 - loss: 2.4028 - accuracy: 0.1016
  7/375 [..............................] - ETA: 3s - loss: 2.3264 - accuracy: 0.1239  
 13/375 [>.............................] - ETA: 3s - loss: 2.2898 - accuracy: 0.1454
 20/375 [>.............................] - ETA: 2s - loss: 2.2518 - accuracy: 0.1871
 26/375 [=>............................] - ETA: 3s - loss: 2.2164 - accuracy: 0.2326
 32/375 [=>............................] - ETA: 2s - loss: 2.1814 - accuracy: 0.2717
 38/375 [==>...........................] - ETA: 2s - loss: 2.1489 - accuracy: 0.3051
 43/375 [==>...........................] - ETA: 3s - loss: 2.1232 - accuracy: 0.3341
 47/375 [==>...........................] - ETA: 3s - loss: 2.1014 - accuracy: 0.3554
 52/375 [===>..........................] - ETA: 3s - loss: 2.0751 - accuracy: 0.3806
 58/375 [===>..........................] - ETA: 3s - loss: 2.0455 - accuracy: 0.4044
 64/375 [====>.........................] - ETA: 3s - loss: 2.0174 - accuracy: 0.4248
 70/375 [====>.........................] - ETA: 3s - loss: 1.9911 - accuracy: 0.4445
 76/375 [=====>........................] - ETA: 2s - loss: 1.9634 - accuracy: 0.4648
 82/375 [=====>........................] - ETA: 2s - loss: 1.9383 - accuracy: 0.4809
 87/375 [=====>........................] - ETA: 2s - loss: 1.9155 - accuracy: 0.4947
 92/375 [======>.......................] - ETA: 2s - loss: 1.8940 - accuracy: 0.5063
 98/375 [======>.......................] - ETA: 2s - loss: 1.8691 - accuracy: 0.5202
104/375 [=======>......................] - ETA: 2s - loss: 1.8463 - accuracy: 0.5322
109/375 [=======>......................] - ETA: 2s - loss: 1.8289 - accuracy: 0.5415
115/375 [========>.....................] - ETA: 2s - loss: 1.8056 - accuracy: 0.5533
121/375 [========>.....................] - ETA: 2s - loss: 1.7828 - accuracy: 0.5640
127/375 [=========>....................] - ETA: 2s - loss: 1.7611 - accuracy: 0.5733
133/375 [=========>....................] - ETA: 2s - loss: 1.7388 - accuracy: 0.5818
139/375 [==========>...................] - ETA: 2s - loss: 1.7161 - accuracy: 0.5912
145/375 [==========>...................] - ETA: 2s - loss: 1.6960 - accuracy: 0.5994
151/375 [===========>..................] - ETA: 2s - loss: 1.6749 - accuracy: 0.6075
157/375 [===========>..................] - ETA: 2s - loss: 1.6562 - accuracy: 0.6147
164/375 [============>.................] - ETA: 2s - loss: 1.6338 - accuracy: 0.6229
170/375 [============>.................] - ETA: 1s - loss: 1.6149 - accuracy: 0.6296
177/375 [=============>................] - ETA: 1s - loss: 1.5950 - accuracy: 0.6361
184/375 [=============>................] - ETA: 1s - loss: 1.5743 - accuracy: 0.6429
190/375 [==============>...............] - ETA: 1s - loss: 1.5579 - accuracy: 0.6479
196/375 [==============>...............] - ETA: 1s - loss: 1.5418 - accuracy: 0.6534
203/375 [===============>..............] - ETA: 1s - loss: 1.5228 - accuracy: 0.6594
209/375 [===============>..............] - ETA: 1s - loss: 1.5071 - accuracy: 0.6642
216/375 [================>.............] - ETA: 1s - loss: 1.4896 - accuracy: 0.6691
222/375 [================>.............] - ETA: 1s - loss: 1.4761 - accuracy: 0.6736
228/375 [=================>............] - ETA: 1s - loss: 1.4618 - accuracy: 0.6778
234/375 [=================>............] - ETA: 1s - loss: 1.4477 - accuracy: 0.6816
240/375 [==================>...........] - ETA: 1s - loss: 1.4341 - accuracy: 0.6854
246/375 [==================>...........] - ETA: 1s - loss: 1.4208 - accuracy: 0.6889
252/375 [===================>..........] - ETA: 1s - loss: 1.4077 - accuracy: 0.6924
256/375 [===================>..........] - ETA: 1s - loss: 1.3997 - accuracy: 0.6944
263/375 [====================>.........] - ETA: 1s - loss: 1.3863 - accuracy: 0.6975
269/375 [====================>.........] - ETA: 0s - loss: 1.3753 - accuracy: 0.7004
276/375 [=====================>........] - ETA: 0s - loss: 1.3616 - accuracy: 0.7039
282/375 [=====================>........] - ETA: 0s - loss: 1.3506 - accuracy: 0.7065
289/375 [======================>.......] - ETA: 0s - loss: 1.3380 - accuracy: 0.7094
295/375 [======================>.......] - ETA: 0s - loss: 1.3263 - accuracy: 0.7122
302/375 [=======================>......] - ETA: 0s - loss: 1.3147 - accuracy: 0.7142
308/375 [=======================>......] - ETA: 0s - loss: 1.3048 - accuracy: 0.7164
314/375 [========================>.....] - ETA: 0s - loss: 1.2949 - accuracy: 0.7187
320/375 [========================>.....] - ETA: 0s - loss: 1.2852 - accuracy: 0.7211
327/375 [=========================>....] - ETA: 0s - loss: 1.2738 - accuracy: 0.7237
333/375 [=========================>....] - ETA: 0s - loss: 1.2654 - accuracy: 0.7254
339/375 [==========================>...] - ETA: 0s - loss: 1.2556 - accuracy: 0.7278
345/375 [==========================>...] - ETA: 0s - loss: 1.2462 - accuracy: 0.7295
351/375 [===========================>..] - ETA: 0s - loss: 1.2371 - accuracy: 0.7318
358/375 [===========================>..] - ETA: 0s - loss: 1.2275 - accuracy: 0.7338
365/375 [============================>.] - ETA: 0s - loss: 1.2172 - accuracy: 0.7360
371/375 [============================>.] - ETA: 0s - loss: 1.2087 - accuracy: 0.7379
375/375 [==============================] - 4s 9ms/step - loss: 1.2033 - accuracy: 0.7391

2022-08-04 14:38:08.506283: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:113] Plugin optimizer for device_type GPU is enabled.


375/375 [==============================] - 5s 13ms/step - loss: 1.2033 - accuracy: 0.7391 - val_loss: 0.6897 - val_accuracy: 0.8541
Epoch 2/10

  1/375 [..............................] - ETA: 13s - loss: 0.6929 - accuracy: 0.8516
  7/375 [..............................] - ETA: 3s - loss: 0.6604 - accuracy: 0.8571 
 14/375 [>.............................] - ETA: 3s - loss: 0.6747 - accuracy: 0.8532
 20/375 [>.............................] - ETA: 3s - loss: 0.6803 - accuracy: 0.8531
 27/375 [=>............................] - ETA: 3s - loss: 0.6850 - accuracy: 0.8527
 34/375 [=>............................] - ETA: 2s - loss: 0.6797 - accuracy: 0.8516
 41/375 [==>...........................] - ETA: 2s - loss: 0.6759 - accuracy: 0.8521
 47/375 [==>...........................] - ETA: 2s - loss: 0.6687 - accuracy: 0.8536
 53/375 [===>..........................] - ETA: 2s - loss: 0.6644 - accuracy: 0.8548
 60/375 [===>..........................] - ETA: 2s - loss: 0.6661 - accuracy: 0.8539
 67/375 [====>.........................] - ETA: 2s - loss: 0.6604 - accuracy: 0.8565
 74/375 [====>.........................] - ETA: 2s - loss: 0.6540 - accuracy: 0.8573
 80/375 [=====>........................] - ETA: 2s - loss: 0.6519 - accuracy: 0.8584
 86/375 [=====>........................] - ETA: 2s - loss: 0.6503 - accuracy: 0.8586
 92/375 [======>.......................] - ETA: 2s - loss: 0.6486 - accuracy: 0.8593
 98/375 [======>.......................] - ETA: 2s - loss: 0.6456 - accuracy: 0.8599
104/375 [=======>......................] - ETA: 2s - loss: 0.6446 - accuracy: 0.8597
110/375 [=======>......................] - ETA: 2s - loss: 0.6438 - accuracy: 0.8602
116/375 [========>.....................] - ETA: 2s - loss: 0.6428 - accuracy: 0.8596
122/375 [========>.....................] - ETA: 2s - loss: 0.6397 - accuracy: 0.8600
128/375 [=========>....................] - ETA: 2s - loss: 0.6380 - accuracy: 0.8595
135/375 [=========>....................] - ETA: 2s - loss: 0.6357 - accuracy: 0.8595
138/375 [==========>...................] - ETA: 2s - loss: 0.6338 - accuracy: 0.8597
143/375 [==========>...................] - ETA: 2s - loss: 0.6319 - accuracy: 0.8596
149/375 [==========>...................] - ETA: 2s - loss: 0.6305 - accuracy: 0.8598
156/375 [===========>..................] - ETA: 1s - loss: 0.6279 - accuracy: 0.8604
163/375 [============>.................] - ETA: 1s - loss: 0.6246 - accuracy: 0.8607
169/375 [============>.................] - ETA: 1s - loss: 0.6225 - accuracy: 0.8605
176/375 [=============>................] - ETA: 1s - loss: 0.6210 - accuracy: 0.8600
183/375 [=============>................] - ETA: 1s - loss: 0.6200 - accuracy: 0.8596
189/375 [==============>...............] - ETA: 1s - loss: 0.6173 - accuracy: 0.8601
195/375 [==============>...............] - ETA: 1s - loss: 0.6154 - accuracy: 0.8607
202/375 [===============>..............] - ETA: 1s - loss: 0.6133 - accuracy: 0.8613
208/375 [===============>..............] - ETA: 1s - loss: 0.6118 - accuracy: 0.8611
214/375 [================>.............] - ETA: 1s - loss: 0.6108 - accuracy: 0.8615
220/375 [================>.............] - ETA: 1s - loss: 0.6084 - accuracy: 0.8624
227/375 [=================>............] - ETA: 1s - loss: 0.6061 - accuracy: 0.8626
233/375 [=================>............] - ETA: 1s - loss: 0.6045 - accuracy: 0.8626
240/375 [==================>...........] - ETA: 1s - loss: 0.6016 - accuracy: 0.8632
246/375 [==================>...........] - ETA: 1s - loss: 0.6008 - accuracy: 0.8631
252/375 [===================>..........] - ETA: 1s - loss: 0.5999 - accuracy: 0.8630
259/375 [===================>..........] - ETA: 1s - loss: 0.5990 - accuracy: 0.8629
265/375 [====================>.........] - ETA: 0s - loss: 0.5981 - accuracy: 0.8630
271/375 [====================>.........] - ETA: 0s - loss: 0.5968 - accuracy: 0.8632
277/375 [=====================>........] - ETA: 0s - loss: 0.5954 - accuracy: 0.8634
284/375 [=====================>........] - ETA: 0s - loss: 0.5928 - accuracy: 0.8640
290/375 [======================>.......] - ETA: 0s - loss: 0.5913 - accuracy: 0.8643
297/375 [======================>.......] - ETA: 0s - loss: 0.5900 - accuracy: 0.8643
303/375 [=======================>......] - ETA: 0s - loss: 0.5894 - accuracy: 0.8640
308/375 [=======================>......] - ETA: 0s - loss: 0.5878 - accuracy: 0.8644
314/375 [========================>.....] - ETA: 0s - loss: 0.5858 - accuracy: 0.8648
321/375 [========================>.....] - ETA: 0s - loss: 0.5842 - accuracy: 0.8651
328/375 [=========================>....] - ETA: 0s - loss: 0.5826 - accuracy: 0.8651
334/375 [=========================>....] - ETA: 0s - loss: 0.5814 - accuracy: 0.8652
340/375 [==========================>...] - ETA: 0s - loss: 0.5800 - accuracy: 0.8653
346/375 [==========================>...] - ETA: 0s - loss: 0.5785 - accuracy: 0.8657
353/375 [===========================>..] - ETA: 0s - loss: 0.5770 - accuracy: 0.8659
358/375 [===========================>..] - ETA: 0s - loss: 0.5762 - accuracy: 0.8657
365/375 [============================>.] - ETA: 0s - loss: 0.5744 - accuracy: 0.8664
371/375 [============================>.] - ETA: 0s - loss: 0.5734 - accuracy: 0.8664
375/375 [==============================] - 3s 9ms/step - loss: 0.5732 - accuracy: 0.8663

375/375 [==============================] - 4s 11ms/step - loss: 0.5732 - accuracy: 0.8663 - val_loss: 0.4882 - val_accuracy: 0.8792
Epoch 3/10

  1/375 [..............................] - ETA: 3s - loss: 0.5550 - accuracy: 0.8359
  8/375 [..............................] - ETA: 3s - loss: 0.4846 - accuracy: 0.8789
 14/375 [>.............................] - ETA: 3s - loss: 0.4843 - accuracy: 0.8806
 21/375 [>.............................] - ETA: 2s - loss: 0.4826 - accuracy: 0.8795
 27/375 [=>............................] - ETA: 2s - loss: 0.4835 - accuracy: 0.8799
 32/375 [=>............................] - ETA: 3s - loss: 0.4846 - accuracy: 0.8796
 38/375 [==>...........................] - ETA: 3s - loss: 0.4824 - accuracy: 0.8808
 44/375 [==>...........................] - ETA: 3s - loss: 0.4848 - accuracy: 0.8784
 50/375 [===>..........................] - ETA: 2s - loss: 0.4816 - accuracy: 0.8795
 56/375 [===>..........................] - ETA: 2s - loss: 0.4825 - accuracy: 0.8792
 62/375 [===>..........................] - ETA: 2s - loss: 0.4802 - accuracy: 0.8803
 68/375 [====>.........................] - ETA: 2s - loss: 0.4791 - accuracy: 0.8796
 74/375 [====>.........................] - ETA: 2s - loss: 0.4847 - accuracy: 0.8777
 80/375 [=====>........................] - ETA: 2s - loss: 0.4826 - accuracy: 0.8790
 87/375 [=====>........................] - ETA: 2s - loss: 0.4784 - accuracy: 0.8816
 93/375 [======>.......................] - ETA: 2s - loss: 0.4790 - accuracy: 0.8817
 99/375 [======>.......................] - ETA: 2s - loss: 0.4775 - accuracy: 0.8816
105/375 [=======>......................] - ETA: 2s - loss: 0.4769 - accuracy: 0.8811
111/375 [=======>......................] - ETA: 2s - loss: 0.4772 - accuracy: 0.8808
117/375 [========>.....................] - ETA: 2s - loss: 0.4769 - accuracy: 0.8812
124/375 [========>.....................] - ETA: 2s - loss: 0.4756 - accuracy: 0.8811
131/375 [=========>....................] - ETA: 2s - loss: 0.4753 - accuracy: 0.8811
137/375 [=========>....................] - ETA: 2s - loss: 0.4749 - accuracy: 0.8810
143/375 [==========>...................] - ETA: 2s - loss: 0.4740 - accuracy: 0.8814
150/375 [===========>..................] - ETA: 1s - loss: 0.4728 - accuracy: 0.8817
156/375 [===========>..................] - ETA: 1s - loss: 0.4725 - accuracy: 0.8813
162/375 [===========>..................] - ETA: 1s - loss: 0.4710 - accuracy: 0.8816
169/375 [============>.................] - ETA: 1s - loss: 0.4709 - accuracy: 0.8817
175/375 [=============>................] - ETA: 1s - loss: 0.4688 - accuracy: 0.8825
181/375 [=============>................] - ETA: 1s - loss: 0.4697 - accuracy: 0.8826
188/375 [==============>...............] - ETA: 1s - loss: 0.4688 - accuracy: 0.8824
194/375 [==============>...............] - ETA: 1s - loss: 0.4683 - accuracy: 0.8826
199/375 [==============>...............] - ETA: 1s - loss: 0.4679 - accuracy: 0.8825
205/375 [===============>..............] - ETA: 1s - loss: 0.4663 - accuracy: 0.8829
210/375 [===============>..............] - ETA: 1s - loss: 0.4664 - accuracy: 0.8830
215/375 [================>.............] - ETA: 1s - loss: 0.4667 - accuracy: 0.8828
221/375 [================>.............] - ETA: 1s - loss: 0.4667 - accuracy: 0.8828
226/375 [=================>............] - ETA: 1s - loss: 0.4662 - accuracy: 0.8829
232/375 [=================>............] - ETA: 1s - loss: 0.4666 - accuracy: 0.8827
239/375 [==================>...........] - ETA: 1s - loss: 0.4659 - accuracy: 0.8828
246/375 [==================>...........] - ETA: 1s - loss: 0.4652 - accuracy: 0.8829
253/375 [===================>..........] - ETA: 1s - loss: 0.4640 - accuracy: 0.8830
259/375 [===================>..........] - ETA: 1s - loss: 0.4626 - accuracy: 0.8833
266/375 [====================>.........] - ETA: 0s - loss: 0.4606 - accuracy: 0.8841
272/375 [====================>.........] - ETA: 0s - loss: 0.4603 - accuracy: 0.8841
279/375 [=====================>........] - ETA: 0s - loss: 0.4603 - accuracy: 0.8843
285/375 [=====================>........] - ETA: 0s - loss: 0.4600 - accuracy: 0.8842
291/375 [======================>.......] - ETA: 0s - loss: 0.4593 - accuracy: 0.8842
298/375 [======================>.......] - ETA: 0s - loss: 0.4597 - accuracy: 0.8840
304/375 [=======================>......] - ETA: 0s - loss: 0.4594 - accuracy: 0.8841
310/375 [=======================>......] - ETA: 0s - loss: 0.4591 - accuracy: 0.8839
316/375 [========================>.....] - ETA: 0s - loss: 0.4582 - accuracy: 0.8839
322/375 [========================>.....] - ETA: 0s - loss: 0.4587 - accuracy: 0.8836
328/375 [=========================>....] - ETA: 0s - loss: 0.4581 - accuracy: 0.8837
334/375 [=========================>....] - ETA: 0s - loss: 0.4576 - accuracy: 0.8837
340/375 [==========================>...] - ETA: 0s - loss: 0.4574 - accuracy: 0.8836
343/375 [==========================>...] - ETA: 0s - loss: 0.4572 - accuracy: 0.8837
350/375 [===========================>..] - ETA: 0s - loss: 0.4559 - accuracy: 0.8841
356/375 [===========================>..] - ETA: 0s - loss: 0.4558 - accuracy: 0.8842
363/375 [============================>.] - ETA: 0s - loss: 0.4551 - accuracy: 0.8842
369/375 [============================>.] - ETA: 0s - loss: 0.4554 - accuracy: 0.8837
375/375 [==============================] - 3s 9ms/step - loss: 0.4545 - accuracy: 0.8841

375/375 [==============================] - 4s 11ms/step - loss: 0.4545 - accuracy: 0.8841 - val_loss: 0.4156 - val_accuracy: 0.8914
Epoch 4/10

  1/375 [..............................] - ETA: 2s - loss: 0.3734 - accuracy: 0.9141
  8/375 [..............................] - ETA: 3s - loss: 0.3809 - accuracy: 0.8906
 14/375 [>.............................] - ETA: 3s - loss: 0.4022 - accuracy: 0.8929
 20/375 [>.............................] - ETA: 3s - loss: 0.3906 - accuracy: 0.8973
 26/375 [=>............................] - ETA: 2s - loss: 0.4012 - accuracy: 0.8945
 33/375 [=>............................] - ETA: 2s - loss: 0.4071 - accuracy: 0.8923
 39/375 [==>...........................] - ETA: 2s - loss: 0.4049 - accuracy: 0.8930
 45/375 [==>...........................] - ETA: 2s - loss: 0.4099 - accuracy: 0.8903
 51/375 [===>..........................] - ETA: 2s - loss: 0.4088 - accuracy: 0.8902
 58/375 [===>..........................] - ETA: 2s - loss: 0.4137 - accuracy: 0.8866
 65/375 [====>.........................] - ETA: 2s - loss: 0.4149 - accuracy: 0.8863
 71/375 [====>.........................] - ETA: 2s - loss: 0.4180 - accuracy: 0.8863
 77/375 [=====>........................] - ETA: 2s - loss: 0.4154 - accuracy: 0.8872
 84/375 [=====>........................] - ETA: 2s - loss: 0.4190 - accuracy: 0.8852
 90/375 [======>.......................] - ETA: 2s - loss: 0.4180 - accuracy: 0.8859
 95/375 [======>.......................] - ETA: 2s - loss: 0.4150 - accuracy: 0.8872
101/375 [=======>......................] - ETA: 2s - loss: 0.4147 - accuracy: 0.8875
107/375 [=======>......................] - ETA: 2s - loss: 0.4150 - accuracy: 0.8879
113/375 [========>.....................] - ETA: 2s - loss: 0.4149 - accuracy: 0.8886
119/375 [========>.....................] - ETA: 2s - loss: 0.4136 - accuracy: 0.8894
124/375 [========>.....................] - ETA: 2s - loss: 0.4129 - accuracy: 0.8899
130/375 [=========>....................] - ETA: 2s - loss: 0.4118 - accuracy: 0.8906
136/375 [=========>....................] - ETA: 2s - loss: 0.4100 - accuracy: 0.8911
142/375 [==========>...................] - ETA: 2s - loss: 0.4080 - accuracy: 0.8918
148/375 [==========>...................] - ETA: 2s - loss: 0.4088 - accuracy: 0.8913
154/375 [===========>..................] - ETA: 1s - loss: 0.4079 - accuracy: 0.8917
161/375 [===========>..................] - ETA: 1s - loss: 0.4089 - accuracy: 0.8907
168/375 [============>.................] - ETA: 1s - loss: 0.4089 - accuracy: 0.8905
172/375 [============>.................] - ETA: 1s - loss: 0.4076 - accuracy: 0.8905
175/375 [=============>................] - ETA: 1s - loss: 0.4069 - accuracy: 0.8906
182/375 [=============>................] - ETA: 1s - loss: 0.4070 - accuracy: 0.8910
188/375 [==============>...............] - ETA: 1s - loss: 0.4059 - accuracy: 0.8913
195/375 [==============>...............] - ETA: 1s - loss: 0.4052 - accuracy: 0.8915
201/375 [===============>..............] - ETA: 1s - loss: 0.4072 - accuracy: 0.8909
207/375 [===============>..............] - ETA: 1s - loss: 0.4069 - accuracy: 0.8913
214/375 [================>.............] - ETA: 1s - loss: 0.4069 - accuracy: 0.8913
220/375 [================>.............] - ETA: 1s - loss: 0.4066 - accuracy: 0.8912
227/375 [=================>............] - ETA: 1s - loss: 0.4070 - accuracy: 0.8910
233/375 [=================>............] - ETA: 1s - loss: 0.4062 - accuracy: 0.8911
239/375 [==================>...........] - ETA: 1s - loss: 0.4059 - accuracy: 0.8911
246/375 [==================>...........] - ETA: 1s - loss: 0.4060 - accuracy: 0.8911
253/375 [===================>..........] - ETA: 1s - loss: 0.4063 - accuracy: 0.8911
260/375 [===================>..........] - ETA: 1s - loss: 0.4069 - accuracy: 0.8910
267/375 [====================>.........] - ETA: 0s - loss: 0.4064 - accuracy: 0.8915
274/375 [====================>.........] - ETA: 0s - loss: 0.4055 - accuracy: 0.8919
281/375 [=====================>........] - ETA: 0s - loss: 0.4048 - accuracy: 0.8922
287/375 [=====================>........] - ETA: 0s - loss: 0.4046 - accuracy: 0.8922
294/375 [======================>.......] - ETA: 0s - loss: 0.4049 - accuracy: 0.8920
300/375 [=======================>......] - ETA: 0s - loss: 0.4039 - accuracy: 0.8922
303/375 [=======================>......] - ETA: 0s - loss: 0.4041 - accuracy: 0.8923
307/375 [=======================>......] - ETA: 0s - loss: 0.4037 - accuracy: 0.8924
314/375 [========================>.....] - ETA: 0s - loss: 0.4037 - accuracy: 0.8925
320/375 [========================>.....] - ETA: 0s - loss: 0.4033 - accuracy: 0.8927
327/375 [=========================>....] - ETA: 0s - loss: 0.4029 - accuracy: 0.8930
331/375 [=========================>....] - ETA: 0s - loss: 0.4025 - accuracy: 0.8932
337/375 [=========================>....] - ETA: 0s - loss: 0.4024 - accuracy: 0.8935
343/375 [==========================>...] - ETA: 0s - loss: 0.4020 - accuracy: 0.8935
349/375 [==========================>...] - ETA: 0s - loss: 0.4019 - accuracy: 0.8934
355/375 [===========================>..] - ETA: 0s - loss: 0.4024 - accuracy: 0.8935
361/375 [===========================>..] - ETA: 0s - loss: 0.4015 - accuracy: 0.8937
367/375 [============================>.] - ETA: 0s - loss: 0.4016 - accuracy: 0.8937
373/375 [============================>.] - ETA: 0s - loss: 0.4019 - accuracy: 0.8933
375/375 [==============================] - 3s 9ms/step - loss: 0.4016 - accuracy: 0.8933

375/375 [==============================] - 4s 12ms/step - loss: 0.4016 - accuracy: 0.8933 - val_loss: 0.3765 - val_accuracy: 0.8993
Epoch 5/10

  1/375 [..............................] - ETA: 2s - loss: 0.3209 - accuracy: 0.9297
  7/375 [..............................] - ETA: 3s - loss: 0.3755 - accuracy: 0.8996
 13/375 [>.............................] - ETA: 3s - loss: 0.3796 - accuracy: 0.8948
 19/375 [>.............................] - ETA: 3s - loss: 0.3710 - accuracy: 0.9005
 24/375 [>.............................] - ETA: 3s - loss: 0.3701 - accuracy: 0.8991
 30/375 [=>............................] - ETA: 3s - loss: 0.3670 - accuracy: 0.8997
 36/375 [=>............................] - ETA: 3s - loss: 0.3638 - accuracy: 0.9019
 42/375 [==>...........................] - ETA: 3s - loss: 0.3711 - accuracy: 0.8996
 49/375 [==>...........................] - ETA: 2s - loss: 0.3692 - accuracy: 0.8992
 56/375 [===>..........................] - ETA: 2s - loss: 0.3714 - accuracy: 0.8980
 63/375 [====>.........................] - ETA: 2s - loss: 0.3699 - accuracy: 0.8983
 69/375 [====>.........................] - ETA: 2s - loss: 0.3730 - accuracy: 0.8975
 76/375 [=====>........................] - ETA: 2s - loss: 0.3732 - accuracy: 0.8974
 83/375 [=====>........................] - ETA: 2s - loss: 0.3741 - accuracy: 0.8961
 90/375 [======>.......................] - ETA: 2s - loss: 0.3749 - accuracy: 0.8964
 97/375 [======>.......................] - ETA: 2s - loss: 0.3759 - accuracy: 0.8963
104/375 [=======>......................] - ETA: 2s - loss: 0.3759 - accuracy: 0.8969
111/375 [=======>......................] - ETA: 2s - loss: 0.3751 - accuracy: 0.8972
117/375 [========>.....................] - ETA: 2s - loss: 0.3739 - accuracy: 0.8974
123/375 [========>.....................] - ETA: 2s - loss: 0.3743 - accuracy: 0.8974
129/375 [=========>....................] - ETA: 2s - loss: 0.3742 - accuracy: 0.8970
135/375 [=========>....................] - ETA: 2s - loss: 0.3763 - accuracy: 0.8966
142/375 [==========>...................] - ETA: 1s - loss: 0.3740 - accuracy: 0.8976
148/375 [==========>...................] - ETA: 1s - loss: 0.3728 - accuracy: 0.8981
155/375 [===========>..................] - ETA: 1s - loss: 0.3732 - accuracy: 0.8978
162/375 [===========>..................] - ETA: 1s - loss: 0.3736 - accuracy: 0.8976
168/375 [============>.................] - ETA: 1s - loss: 0.3758 - accuracy: 0.8969
174/375 [============>.................] - ETA: 1s - loss: 0.3765 - accuracy: 0.8966
180/375 [=============>................] - ETA: 1s - loss: 0.3758 - accuracy: 0.8971
186/375 [=============>................] - ETA: 1s - loss: 0.3751 - accuracy: 0.8972
192/375 [==============>...............] - ETA: 1s - loss: 0.3743 - accuracy: 0.8973
198/375 [==============>...............] - ETA: 1s - loss: 0.3745 - accuracy: 0.8974
204/375 [===============>..............] - ETA: 1s - loss: 0.3755 - accuracy: 0.8970
210/375 [===============>..............] - ETA: 1s - loss: 0.3770 - accuracy: 0.8967
217/375 [================>.............] - ETA: 1s - loss: 0.3780 - accuracy: 0.8960
223/375 [================>.............] - ETA: 1s - loss: 0.3770 - accuracy: 0.8964
230/375 [=================>............] - ETA: 1s - loss: 0.3766 - accuracy: 0.8962
236/375 [=================>............] - ETA: 1s - loss: 0.3775 - accuracy: 0.8960
243/375 [==================>...........] - ETA: 1s - loss: 0.3760 - accuracy: 0.8968
249/375 [==================>...........] - ETA: 1s - loss: 0.3759 - accuracy: 0.8965
256/375 [===================>..........] - ETA: 1s - loss: 0.3760 - accuracy: 0.8965
263/375 [====================>.........] - ETA: 0s - loss: 0.3751 - accuracy: 0.8970
269/375 [====================>.........] - ETA: 0s - loss: 0.3749 - accuracy: 0.8971
276/375 [=====================>........] - ETA: 0s - loss: 0.3741 - accuracy: 0.8976
283/375 [=====================>........] - ETA: 0s - loss: 0.3737 - accuracy: 0.8977
289/375 [======================>.......] - ETA: 0s - loss: 0.3734 - accuracy: 0.8980
295/375 [======================>.......] - ETA: 0s - loss: 0.3724 - accuracy: 0.8984
301/375 [=======================>......] - ETA: 0s - loss: 0.3717 - accuracy: 0.8987
307/375 [=======================>......] - ETA: 0s - loss: 0.3708 - accuracy: 0.8991
313/375 [========================>.....] - ETA: 0s - loss: 0.3710 - accuracy: 0.8991
319/375 [========================>.....] - ETA: 0s - loss: 0.3708 - accuracy: 0.8992
325/375 [=========================>....] - ETA: 0s - loss: 0.3712 - accuracy: 0.8989
331/375 [=========================>....] - ETA: 0s - loss: 0.3711 - accuracy: 0.8987
337/375 [=========================>....] - ETA: 0s - loss: 0.3713 - accuracy: 0.8989
343/375 [==========================>...] - ETA: 0s - loss: 0.3709 - accuracy: 0.8990
349/375 [==========================>...] - ETA: 0s - loss: 0.3707 - accuracy: 0.8990
356/375 [===========================>..] - ETA: 0s - loss: 0.3708 - accuracy: 0.8992
362/375 [===========================>..] - ETA: 0s - loss: 0.3704 - accuracy: 0.8993
369/375 [============================>.] - ETA: 0s - loss: 0.3695 - accuracy: 0.8999
375/375 [==============================] - 3s 9ms/step - loss: 0.3697 - accuracy: 0.8998

375/375 [==============================] - 4s 11ms/step - loss: 0.3697 - accuracy: 0.8998 - val_loss: 0.3512 - val_accuracy: 0.9028
Epoch 6/10

  1/375 [..............................] - ETA: 3s - loss: 0.3494 - accuracy: 0.8750
  7/375 [..............................] - ETA: 3s - loss: 0.3411 - accuracy: 0.9063
 13/375 [>.............................] - ETA: 3s - loss: 0.3662 - accuracy: 0.9008
 19/375 [>.............................] - ETA: 3s - loss: 0.3568 - accuracy: 0.9009
 25/375 [=>............................] - ETA: 3s - loss: 0.3476 - accuracy: 0.9031
 31/375 [=>............................] - ETA: 3s - loss: 0.3415 - accuracy: 0.9055
 36/375 [=>............................] - ETA: 3s - loss: 0.3474 - accuracy: 0.9056
 42/375 [==>...........................] - ETA: 3s - loss: 0.3475 - accuracy: 0.9055
 49/375 [==>...........................] - ETA: 2s - loss: 0.3532 - accuracy: 0.9043
 55/375 [===>..........................] - ETA: 2s - loss: 0.3508 - accuracy: 0.9044
 61/375 [===>..........................] - ETA: 2s - loss: 0.3517 - accuracy: 0.9045
 68/375 [====>.........................] - ETA: 2s - loss: 0.3541 - accuracy: 0.9028
 74/375 [====>.........................] - ETA: 2s - loss: 0.3575 - accuracy: 0.9020
 80/375 [=====>........................] - ETA: 2s - loss: 0.3595 - accuracy: 0.9015
 87/375 [=====>........................] - ETA: 2s - loss: 0.3590 - accuracy: 0.9020
 93/375 [======>.......................] - ETA: 2s - loss: 0.3582 - accuracy: 0.9027
100/375 [=======>......................] - ETA: 2s - loss: 0.3600 - accuracy: 0.9023
107/375 [=======>......................] - ETA: 2s - loss: 0.3595 - accuracy: 0.9030
113/375 [========>.....................] - ETA: 2s - loss: 0.3590 - accuracy: 0.9029
120/375 [========>.....................] - ETA: 2s - loss: 0.3571 - accuracy: 0.9032
126/375 [=========>....................] - ETA: 2s - loss: 0.3575 - accuracy: 0.9027
133/375 [=========>....................] - ETA: 2s - loss: 0.3574 - accuracy: 0.9025
137/375 [=========>....................] - ETA: 2s - loss: 0.3568 - accuracy: 0.9029
140/375 [==========>...................] - ETA: 2s - loss: 0.3555 - accuracy: 0.9030
146/375 [==========>...................] - ETA: 2s - loss: 0.3540 - accuracy: 0.9037
153/375 [===========>..................] - ETA: 1s - loss: 0.3530 - accuracy: 0.9038
159/375 [===========>..................] - ETA: 1s - loss: 0.3536 - accuracy: 0.9036
165/375 [============>.................] - ETA: 1s - loss: 0.3530 - accuracy: 0.9042
172/375 [============>.................] - ETA: 1s - loss: 0.3538 - accuracy: 0.9042
178/375 [=============>................] - ETA: 1s - loss: 0.3531 - accuracy: 0.9045
184/375 [=============>................] - ETA: 1s - loss: 0.3521 - accuracy: 0.9048
190/375 [==============>...............] - ETA: 1s - loss: 0.3525 - accuracy: 0.9046
196/375 [==============>...............] - ETA: 1s - loss: 0.3532 - accuracy: 0.9045
203/375 [===============>..............] - ETA: 1s - loss: 0.3517 - accuracy: 0.9047
210/375 [===============>..............] - ETA: 1s - loss: 0.3513 - accuracy: 0.9049
217/375 [================>.............] - ETA: 1s - loss: 0.3503 - accuracy: 0.9050
223/375 [================>.............] - ETA: 1s - loss: 0.3509 - accuracy: 0.9047
229/375 [=================>............] - ETA: 1s - loss: 0.3510 - accuracy: 0.9043
236/375 [=================>............] - ETA: 1s - loss: 0.3509 - accuracy: 0.9045
243/375 [==================>...........] - ETA: 1s - loss: 0.3509 - accuracy: 0.9043
249/375 [==================>...........] - ETA: 1s - loss: 0.3505 - accuracy: 0.9045
256/375 [===================>..........] - ETA: 1s - loss: 0.3499 - accuracy: 0.9046
262/375 [===================>..........] - ETA: 0s - loss: 0.3500 - accuracy: 0.9044
268/375 [====================>.........] - ETA: 0s - loss: 0.3504 - accuracy: 0.9041
275/375 [=====================>........] - ETA: 0s - loss: 0.3504 - accuracy: 0.9043
281/375 [=====================>........] - ETA: 0s - loss: 0.3492 - accuracy: 0.9046
288/375 [======================>.......] - ETA: 0s - loss: 0.3490 - accuracy: 0.9048
295/375 [======================>.......] - ETA: 0s - loss: 0.3483 - accuracy: 0.9052
301/375 [=======================>......] - ETA: 0s - loss: 0.3484 - accuracy: 0.9053
307/375 [=======================>......] - ETA: 0s - loss: 0.3478 - accuracy: 0.9056
314/375 [========================>.....] - ETA: 0s - loss: 0.3472 - accuracy: 0.9059
320/375 [========================>.....] - ETA: 0s - loss: 0.3471 - accuracy: 0.9060
326/375 [=========================>....] - ETA: 0s - loss: 0.3473 - accuracy: 0.9057
333/375 [=========================>....] - ETA: 0s - loss: 0.3475 - accuracy: 0.9057
340/375 [==========================>...] - ETA: 0s - loss: 0.3472 - accuracy: 0.9059
347/375 [==========================>...] - ETA: 0s - loss: 0.3480 - accuracy: 0.9054
354/375 [===========================>..] - ETA: 0s - loss: 0.3479 - accuracy: 0.9056
360/375 [===========================>..] - ETA: 0s - loss: 0.3481 - accuracy: 0.9055
367/375 [============================>.] - ETA: 0s - loss: 0.3480 - accuracy: 0.9054
374/375 [============================>.] - ETA: 0s - loss: 0.3476 - accuracy: 0.9055
375/375 [==============================] - 3s 9ms/step - loss: 0.3477 - accuracy: 0.9055

375/375 [==============================] - 4s 11ms/step - loss: 0.3477 - accuracy: 0.9055 - val_loss: 0.3326 - val_accuracy: 0.9065
Epoch 7/10

  1/375 [..............................] - ETA: 3s - loss: 0.2745 - accuracy: 0.9141
  7/375 [..............................] - ETA: 3s - loss: 0.3181 - accuracy: 0.9096
 14/375 [>.............................] - ETA: 3s - loss: 0.3302 - accuracy: 0.9040
 19/375 [>.............................] - ETA: 3s - loss: 0.3210 - accuracy: 0.9083
 23/375 [>.............................] - ETA: 3s - loss: 0.3206 - accuracy: 0.9090
 29/375 [=>............................] - ETA: 3s - loss: 0.3157 - accuracy: 0.9098
 36/375 [=>............................] - ETA: 3s - loss: 0.3172 - accuracy: 0.9099
 42/375 [==>...........................] - ETA: 3s - loss: 0.3196 - accuracy: 0.9102
 48/375 [==>...........................] - ETA: 3s - loss: 0.3219 - accuracy: 0.9103
 54/375 [===>..........................] - ETA: 2s - loss: 0.3203 - accuracy: 0.9115
 61/375 [===>..........................] - ETA: 2s - loss: 0.3228 - accuracy: 0.9110
 68/375 [====>.........................] - ETA: 2s - loss: 0.3209 - accuracy: 0.9115
 75/375 [=====>........................] - ETA: 2s - loss: 0.3245 - accuracy: 0.9111
 82/375 [=====>........................] - ETA: 2s - loss: 0.3266 - accuracy: 0.9111
 89/375 [======>.......................] - ETA: 2s - loss: 0.3248 - accuracy: 0.9115
 95/375 [======>.......................] - ETA: 2s - loss: 0.3266 - accuracy: 0.9108
102/375 [=======>......................] - ETA: 2s - loss: 0.3304 - accuracy: 0.9105
108/375 [=======>......................] - ETA: 2s - loss: 0.3292 - accuracy: 0.9102
114/375 [========>.....................] - ETA: 2s - loss: 0.3303 - accuracy: 0.9097
121/375 [========>.....................] - ETA: 2s - loss: 0.3300 - accuracy: 0.9100
127/375 [=========>....................] - ETA: 2s - loss: 0.3273 - accuracy: 0.9107
134/375 [=========>....................] - ETA: 2s - loss: 0.3252 - accuracy: 0.9112
138/375 [==========>...................] - ETA: 2s - loss: 0.3264 - accuracy: 0.9107
145/375 [==========>...................] - ETA: 2s - loss: 0.3269 - accuracy: 0.9106
152/375 [===========>..................] - ETA: 1s - loss: 0.3264 - accuracy: 0.9103
158/375 [===========>..................] - ETA: 1s - loss: 0.3263 - accuracy: 0.9108
165/375 [============>.................] - ETA: 1s - loss: 0.3258 - accuracy: 0.9105
171/375 [============>.................] - ETA: 1s - loss: 0.3281 - accuracy: 0.9100
178/375 [=============>................] - ETA: 1s - loss: 0.3289 - accuracy: 0.9101
185/375 [=============>................] - ETA: 1s - loss: 0.3292 - accuracy: 0.9099
192/375 [==============>...............] - ETA: 1s - loss: 0.3288 - accuracy: 0.9097
199/375 [==============>...............] - ETA: 1s - loss: 0.3271 - accuracy: 0.9105
206/375 [===============>..............] - ETA: 1s - loss: 0.3274 - accuracy: 0.9104
213/375 [================>.............] - ETA: 1s - loss: 0.3273 - accuracy: 0.9102
219/375 [================>.............] - ETA: 1s - loss: 0.3277 - accuracy: 0.9099
226/375 [=================>............] - ETA: 1s - loss: 0.3283 - accuracy: 0.9097
232/375 [=================>............] - ETA: 1s - loss: 0.3295 - accuracy: 0.9091
238/375 [==================>...........] - ETA: 1s - loss: 0.3309 - accuracy: 0.9086
244/375 [==================>...........] - ETA: 1s - loss: 0.3312 - accuracy: 0.9084
251/375 [===================>..........] - ETA: 1s - loss: 0.3332 - accuracy: 0.9077
257/375 [===================>..........] - ETA: 1s - loss: 0.3328 - accuracy: 0.9079
264/375 [====================>.........] - ETA: 0s - loss: 0.3328 - accuracy: 0.9082
271/375 [====================>.........] - ETA: 0s - loss: 0.3327 - accuracy: 0.9084
278/375 [=====================>........] - ETA: 0s - loss: 0.3325 - accuracy: 0.9086
284/375 [=====================>........] - ETA: 0s - loss: 0.3332 - accuracy: 0.9083
290/375 [======================>.......] - ETA: 0s - loss: 0.3319 - accuracy: 0.9089
297/375 [======================>.......] - ETA: 0s - loss: 0.3323 - accuracy: 0.9089
303/375 [=======================>......] - ETA: 0s - loss: 0.3317 - accuracy: 0.9091
309/375 [=======================>......] - ETA: 0s - loss: 0.3313 - accuracy: 0.9091
315/375 [========================>.....] - ETA: 0s - loss: 0.3313 - accuracy: 0.9090
321/375 [========================>.....] - ETA: 0s - loss: 0.3322 - accuracy: 0.9088
327/375 [=========================>....] - ETA: 0s - loss: 0.3315 - accuracy: 0.9091
334/375 [=========================>....] - ETA: 0s - loss: 0.3313 - accuracy: 0.9092
341/375 [==========================>...] - ETA: 0s - loss: 0.3313 - accuracy: 0.9092
347/375 [==========================>...] - ETA: 0s - loss: 0.3305 - accuracy: 0.9094
353/375 [===========================>..] - ETA: 0s - loss: 0.3300 - accuracy: 0.9096
359/375 [===========================>..] - ETA: 0s - loss: 0.3303 - accuracy: 0.9094
365/375 [============================>.] - ETA: 0s - loss: 0.3303 - accuracy: 0.9095
370/375 [============================>.] - ETA: 0s - loss: 0.3304 - accuracy: 0.9095
375/375 [==============================] - 3s 9ms/step - loss: 0.3307 - accuracy: 0.9094

375/375 [==============================] - 4s 11ms/step - loss: 0.3307 - accuracy: 0.9094 - val_loss: 0.3178 - val_accuracy: 0.9092
Epoch 8/10

  1/375 [..............................] - ETA: 3s - loss: 0.3349 - accuracy: 0.9375
  7/375 [..............................] - ETA: 3s - loss: 0.3441 - accuracy: 0.9096
 14/375 [>.............................] - ETA: 2s - loss: 0.3398 - accuracy: 0.9102
 21/375 [>.............................] - ETA: 2s - loss: 0.3437 - accuracy: 0.9044
 27/375 [=>............................] - ETA: 2s - loss: 0.3359 - accuracy: 0.9060
 33/375 [=>............................] - ETA: 2s - loss: 0.3294 - accuracy: 0.9081
 39/375 [==>...........................] - ETA: 2s - loss: 0.3328 - accuracy: 0.9075
 45/375 [==>...........................] - ETA: 2s - loss: 0.3317 - accuracy: 0.9080
 51/375 [===>..........................] - ETA: 2s - loss: 0.3283 - accuracy: 0.9085
 58/375 [===>..........................] - ETA: 2s - loss: 0.3293 - accuracy: 0.9092
 65/375 [====>.........................] - ETA: 2s - loss: 0.3259 - accuracy: 0.9094
 71/375 [====>.........................] - ETA: 2s - loss: 0.3240 - accuracy: 0.9096
 78/375 [=====>........................] - ETA: 2s - loss: 0.3253 - accuracy: 0.9091
 84/375 [=====>........................] - ETA: 2s - loss: 0.3257 - accuracy: 0.9083
 91/375 [======>.......................] - ETA: 2s - loss: 0.3267 - accuracy: 0.9085
 97/375 [======>.......................] - ETA: 2s - loss: 0.3268 - accuracy: 0.9084
103/375 [=======>......................] - ETA: 2s - loss: 0.3269 - accuracy: 0.9084
110/375 [=======>......................] - ETA: 2s - loss: 0.3273 - accuracy: 0.9080
117/375 [========>.....................] - ETA: 2s - loss: 0.3261 - accuracy: 0.9084
123/375 [========>.....................] - ETA: 2s - loss: 0.3243 - accuracy: 0.9090
130/375 [=========>....................] - ETA: 2s - loss: 0.3232 - accuracy: 0.9096
136/375 [=========>....................] - ETA: 2s - loss: 0.3223 - accuracy: 0.9101
143/375 [==========>...................] - ETA: 1s - loss: 0.3223 - accuracy: 0.9100
150/375 [===========>..................] - ETA: 1s - loss: 0.3208 - accuracy: 0.9103
157/375 [===========>..................] - ETA: 1s - loss: 0.3205 - accuracy: 0.9105
163/375 [============>.................] - ETA: 1s - loss: 0.3200 - accuracy: 0.9111
170/375 [============>.................] - ETA: 1s - loss: 0.3212 - accuracy: 0.9109
175/375 [=============>................] - ETA: 1s - loss: 0.3204 - accuracy: 0.9108
181/375 [=============>................] - ETA: 1s - loss: 0.3215 - accuracy: 0.9106
188/375 [==============>...............] - ETA: 1s - loss: 0.3210 - accuracy: 0.9112
195/375 [==============>...............] - ETA: 1s - loss: 0.3208 - accuracy: 0.9111
202/375 [===============>..............] - ETA: 1s - loss: 0.3205 - accuracy: 0.9113
209/375 [===============>..............] - ETA: 1s - loss: 0.3206 - accuracy: 0.9115
216/375 [================>.............] - ETA: 1s - loss: 0.3205 - accuracy: 0.9115
223/375 [================>.............] - ETA: 1s - loss: 0.3191 - accuracy: 0.9118
229/375 [=================>............] - ETA: 1s - loss: 0.3198 - accuracy: 0.9116
235/375 [=================>............] - ETA: 1s - loss: 0.3186 - accuracy: 0.9122
241/375 [==================>...........] - ETA: 1s - loss: 0.3190 - accuracy: 0.9121
247/375 [==================>...........] - ETA: 1s - loss: 0.3189 - accuracy: 0.9121
253/375 [===================>..........] - ETA: 1s - loss: 0.3181 - accuracy: 0.9124
260/375 [===================>..........] - ETA: 0s - loss: 0.3174 - accuracy: 0.9126
266/375 [====================>.........] - ETA: 0s - loss: 0.3179 - accuracy: 0.9124
272/375 [====================>.........] - ETA: 0s - loss: 0.3181 - accuracy: 0.9122
279/375 [=====================>........] - ETA: 0s - loss: 0.3179 - accuracy: 0.9124
286/375 [=====================>........] - ETA: 0s - loss: 0.3177 - accuracy: 0.9126
290/375 [======================>.......] - ETA: 0s - loss: 0.3172 - accuracy: 0.9127
297/375 [======================>.......] - ETA: 0s - loss: 0.3180 - accuracy: 0.9124
303/375 [=======================>......] - ETA: 0s - loss: 0.3172 - accuracy: 0.9128
309/375 [=======================>......] - ETA: 0s - loss: 0.3164 - accuracy: 0.9129
316/375 [========================>.....] - ETA: 0s - loss: 0.3167 - accuracy: 0.9128
322/375 [========================>.....] - ETA: 0s - loss: 0.3163 - accuracy: 0.9131
329/375 [=========================>....] - ETA: 0s - loss: 0.3159 - accuracy: 0.9132
335/375 [=========================>....] - ETA: 0s - loss: 0.3155 - accuracy: 0.9135
342/375 [==========================>...] - ETA: 0s - loss: 0.3161 - accuracy: 0.9134
347/375 [==========================>...] - ETA: 0s - loss: 0.3160 - accuracy: 0.9135
354/375 [===========================>..] - ETA: 0s - loss: 0.3170 - accuracy: 0.9133
361/375 [===========================>..] - ETA: 0s - loss: 0.3169 - accuracy: 0.9133
368/375 [============================>.] - ETA: 0s - loss: 0.3165 - accuracy: 0.9133
374/375 [============================>.] - ETA: 0s - loss: 0.3172 - accuracy: 0.9131
375/375 [==============================] - 3s 8ms/step - loss: 0.3170 - accuracy: 0.9132

375/375 [==============================] - 4s 11ms/step - loss: 0.3170 - accuracy: 0.9132 - val_loss: 0.3057 - val_accuracy: 0.9118
Epoch 9/10

  1/375 [..............................] - ETA: 3s - loss: 0.2873 - accuracy: 0.9219
  7/375 [..............................] - ETA: 3s - loss: 0.3191 - accuracy: 0.9040
 13/375 [>.............................] - ETA: 3s - loss: 0.3128 - accuracy: 0.9056
 19/375 [>.............................] - ETA: 3s - loss: 0.3129 - accuracy: 0.9067
 26/375 [=>............................] - ETA: 3s - loss: 0.3133 - accuracy: 0.9072
 31/375 [=>............................] - ETA: 3s - loss: 0.3120 - accuracy: 0.9083
 37/375 [=>............................] - ETA: 3s - loss: 0.3160 - accuracy: 0.9082
 43/375 [==>...........................] - ETA: 3s - loss: 0.3126 - accuracy: 0.9104
 49/375 [==>...........................] - ETA: 2s - loss: 0.3115 - accuracy: 0.9094
 55/375 [===>..........................] - ETA: 2s - loss: 0.3145 - accuracy: 0.9092
 61/375 [===>..........................] - ETA: 2s - loss: 0.3126 - accuracy: 0.9110
 67/375 [====>.........................] - ETA: 2s - loss: 0.3154 - accuracy: 0.9110
 74/375 [====>.........................] - ETA: 2s - loss: 0.3146 - accuracy: 0.9106
 81/375 [=====>........................] - ETA: 2s - loss: 0.3133 - accuracy: 0.9110
 87/375 [=====>........................] - ETA: 2s - loss: 0.3095 - accuracy: 0.9115
 93/375 [======>.......................] - ETA: 2s - loss: 0.3073 - accuracy: 0.9122
 99/375 [======>.......................] - ETA: 2s - loss: 0.3074 - accuracy: 0.9126
106/375 [=======>......................] - ETA: 2s - loss: 0.3068 - accuracy: 0.9133
113/375 [========>.....................] - ETA: 2s - loss: 0.3075 - accuracy: 0.9128
120/375 [========>.....................] - ETA: 2s - loss: 0.3085 - accuracy: 0.9120
126/375 [=========>....................] - ETA: 2s - loss: 0.3076 - accuracy: 0.9126
132/375 [=========>....................] - ETA: 2s - loss: 0.3084 - accuracy: 0.9125
138/375 [==========>...................] - ETA: 2s - loss: 0.3077 - accuracy: 0.9135
145/375 [==========>...................] - ETA: 1s - loss: 0.3101 - accuracy: 0.9129
151/375 [===========>..................] - ETA: 1s - loss: 0.3101 - accuracy: 0.9131
158/375 [===========>..................] - ETA: 1s - loss: 0.3108 - accuracy: 0.9130
164/375 [============>.................] - ETA: 1s - loss: 0.3107 - accuracy: 0.9130
170/375 [============>.................] - ETA: 1s - loss: 0.3089 - accuracy: 0.9138
177/375 [=============>................] - ETA: 1s - loss: 0.3087 - accuracy: 0.9143
183/375 [=============>................] - ETA: 1s - loss: 0.3085 - accuracy: 0.9142
187/375 [=============>................] - ETA: 1s - loss: 0.3088 - accuracy: 0.9140
193/375 [==============>...............] - ETA: 1s - loss: 0.3096 - accuracy: 0.9144
199/375 [==============>...............] - ETA: 1s - loss: 0.3112 - accuracy: 0.9141
205/375 [===============>..............] - ETA: 1s - loss: 0.3112 - accuracy: 0.9145
211/375 [===============>..............] - ETA: 1s - loss: 0.3113 - accuracy: 0.9144
217/375 [================>.............] - ETA: 1s - loss: 0.3098 - accuracy: 0.9149
224/375 [================>.............] - ETA: 1s - loss: 0.3097 - accuracy: 0.9148
230/375 [=================>............] - ETA: 1s - loss: 0.3092 - accuracy: 0.9148
232/375 [=================>............] - ETA: 1s - loss: 0.3089 - accuracy: 0.9150
238/375 [==================>...........] - ETA: 1s - loss: 0.3080 - accuracy: 0.9157
244/375 [==================>...........] - ETA: 1s - loss: 0.3075 - accuracy: 0.9157
250/375 [===================>..........] - ETA: 1s - loss: 0.3067 - accuracy: 0.9158
256/375 [===================>..........] - ETA: 1s - loss: 0.3066 - accuracy: 0.9159
262/375 [===================>..........] - ETA: 1s - loss: 0.3057 - accuracy: 0.9163
268/375 [====================>.........] - ETA: 0s - loss: 0.3055 - accuracy: 0.9164
274/375 [====================>.........] - ETA: 0s - loss: 0.3068 - accuracy: 0.9161
280/375 [=====================>........] - ETA: 0s - loss: 0.3061 - accuracy: 0.9163
286/375 [=====================>........] - ETA: 0s - loss: 0.3070 - accuracy: 0.9160
292/375 [======================>.......] - ETA: 0s - loss: 0.3068 - accuracy: 0.9161
298/375 [======================>.......] - ETA: 0s - loss: 0.3069 - accuracy: 0.9159
304/375 [=======================>......] - ETA: 0s - loss: 0.3071 - accuracy: 0.9157
310/375 [=======================>......] - ETA: 0s - loss: 0.3081 - accuracy: 0.9153
316/375 [========================>.....] - ETA: 0s - loss: 0.3078 - accuracy: 0.9153
322/375 [========================>.....] - ETA: 0s - loss: 0.3081 - accuracy: 0.9152
328/375 [=========================>....] - ETA: 0s - loss: 0.3083 - accuracy: 0.9152
334/375 [=========================>....] - ETA: 0s - loss: 0.3078 - accuracy: 0.9153
340/375 [==========================>...] - ETA: 0s - loss: 0.3067 - accuracy: 0.9157
346/375 [==========================>...] - ETA: 0s - loss: 0.3069 - accuracy: 0.9155
353/375 [===========================>..] - ETA: 0s - loss: 0.3063 - accuracy: 0.9159
357/375 [===========================>..] - ETA: 0s - loss: 0.3063 - accuracy: 0.9160
363/375 [============================>.] - ETA: 0s - loss: 0.3061 - accuracy: 0.9161
369/375 [============================>.] - ETA: 0s - loss: 0.3052 - accuracy: 0.9163
375/375 [==============================] - 3s 9ms/step - loss: 0.3052 - accuracy: 0.9160

375/375 [==============================] - 4s 12ms/step - loss: 0.3052 - accuracy: 0.9160 - val_loss: 0.2957 - val_accuracy: 0.9151
Epoch 10/10

  1/375 [..............................] - ETA: 2s - loss: 0.4804 - accuracy: 0.8594
  7/375 [..............................] - ETA: 3s - loss: 0.3394 - accuracy: 0.9129
 13/375 [>.............................] - ETA: 3s - loss: 0.3249 - accuracy: 0.9117
 19/375 [>.............................] - ETA: 3s - loss: 0.3163 - accuracy: 0.9149
 25/375 [=>............................] - ETA: 3s - loss: 0.3210 - accuracy: 0.9106
 32/375 [=>............................] - ETA: 2s - loss: 0.3172 - accuracy: 0.9124
 39/375 [==>...........................] - ETA: 2s - loss: 0.3114 - accuracy: 0.9145
 46/375 [==>...........................] - ETA: 2s - loss: 0.3122 - accuracy: 0.9136
 53/375 [===>..........................] - ETA: 2s - loss: 0.3062 - accuracy: 0.9157
 59/375 [===>..........................] - ETA: 2s - loss: 0.3055 - accuracy: 0.9162
 66/375 [====>.........................] - ETA: 2s - loss: 0.3004 - accuracy: 0.9177
 73/375 [====>.........................] - ETA: 2s - loss: 0.3021 - accuracy: 0.9166
 80/375 [=====>........................] - ETA: 2s - loss: 0.3030 - accuracy: 0.9162
 87/375 [=====>........................] - ETA: 2s - loss: 0.3009 - accuracy: 0.9168
 94/375 [======>.......................] - ETA: 2s - loss: 0.3002 - accuracy: 0.9170
101/375 [=======>......................] - ETA: 2s - loss: 0.2982 - accuracy: 0.9175
107/375 [=======>......................] - ETA: 2s - loss: 0.2964 - accuracy: 0.9176
114/375 [========>.....................] - ETA: 2s - loss: 0.2974 - accuracy: 0.9174
120/375 [========>.....................] - ETA: 2s - loss: 0.2959 - accuracy: 0.9175
126/375 [=========>....................] - ETA: 2s - loss: 0.2965 - accuracy: 0.9177
133/375 [=========>....................] - ETA: 2s - loss: 0.2962 - accuracy: 0.9181
139/375 [==========>...................] - ETA: 1s - loss: 0.2955 - accuracy: 0.9182
146/375 [==========>...................] - ETA: 1s - loss: 0.2974 - accuracy: 0.9179
153/375 [===========>..................] - ETA: 1s - loss: 0.2992 - accuracy: 0.9172
157/375 [===========>..................] - ETA: 1s - loss: 0.2990 - accuracy: 0.9168
162/375 [===========>..................] - ETA: 1s - loss: 0.2990 - accuracy: 0.9168
168/375 [============>.................] - ETA: 1s - loss: 0.2981 - accuracy: 0.9171
174/375 [============>.................] - ETA: 1s - loss: 0.2986 - accuracy: 0.9169
181/375 [=============>................] - ETA: 1s - loss: 0.2994 - accuracy: 0.9168
188/375 [==============>...............] - ETA: 1s - loss: 0.2975 - accuracy: 0.9173
194/375 [==============>...............] - ETA: 1s - loss: 0.2954 - accuracy: 0.9180
201/375 [===============>..............] - ETA: 1s - loss: 0.2946 - accuracy: 0.9183
208/375 [===============>..............] - ETA: 1s - loss: 0.2950 - accuracy: 0.9184
215/375 [================>.............] - ETA: 1s - loss: 0.2950 - accuracy: 0.9181
222/375 [================>.............] - ETA: 1s - loss: 0.2960 - accuracy: 0.9178
229/375 [=================>............] - ETA: 1s - loss: 0.2959 - accuracy: 0.9180
236/375 [=================>............] - ETA: 1s - loss: 0.2961 - accuracy: 0.9180
243/375 [==================>...........] - ETA: 1s - loss: 0.2951 - accuracy: 0.9182
249/375 [==================>...........] - ETA: 1s - loss: 0.2947 - accuracy: 0.9183
256/375 [===================>..........] - ETA: 0s - loss: 0.2943 - accuracy: 0.9181
263/375 [====================>.........] - ETA: 0s - loss: 0.2942 - accuracy: 0.9182
270/375 [====================>.........] - ETA: 0s - loss: 0.2951 - accuracy: 0.9179
276/375 [=====================>........] - ETA: 0s - loss: 0.2959 - accuracy: 0.9175
283/375 [=====================>........] - ETA: 0s - loss: 0.2960 - accuracy: 0.9176
289/375 [======================>.......] - ETA: 0s - loss: 0.2957 - accuracy: 0.9178
296/375 [======================>.......] - ETA: 0s - loss: 0.2962 - accuracy: 0.9177
302/375 [=======================>......] - ETA: 0s - loss: 0.2954 - accuracy: 0.9178
308/375 [=======================>......] - ETA: 0s - loss: 0.2955 - accuracy: 0.9179
314/375 [========================>.....] - ETA: 0s - loss: 0.2956 - accuracy: 0.9182
320/375 [========================>.....] - ETA: 0s - loss: 0.2962 - accuracy: 0.9181
327/375 [=========================>....] - ETA: 0s - loss: 0.2954 - accuracy: 0.9184
333/375 [=========================>....] - ETA: 0s - loss: 0.2955 - accuracy: 0.9184
339/375 [==========================>...] - ETA: 0s - loss: 0.2950 - accuracy: 0.9184
345/375 [==========================>...] - ETA: 0s - loss: 0.2953 - accuracy: 0.9183
352/375 [===========================>..] - ETA: 0s - loss: 0.2950 - accuracy: 0.9183
358/375 [===========================>..] - ETA: 0s - loss: 0.2949 - accuracy: 0.9184
364/375 [============================>.] - ETA: 0s - loss: 0.2946 - accuracy: 0.9185
370/375 [============================>.] - ETA: 0s - loss: 0.2949 - accuracy: 0.9185
375/375 [==============================] - 3s 8ms/step - loss: 0.2951 - accuracy: 0.9184

375/375 [==============================] - 4s 11ms/step - loss: 0.2951 - accuracy: 0.9184 - val_loss: 0.2864 - val_accuracy: 0.9176

b128_sgd_hist <- history
plot(b128_sgd_hist)

plot(b16_hist)

Pick an Adaptive Learning Rate Optimizer

Once we’ve picked an adequate batch size, next we want to start tuning the learning rate. There are two main considerations:

1. We usually always assess different learning rates ranging from [1e-1, 1e-7].
2. Adaptive learning rate optimizers nearly always outperform regular SGD

Adaptive learning rates help to escape “saddle points”.

Tips: the primary adaptive learning rates include:

• SGD with momentum ?optimizer_sgd().
• RMSprop (?optimizer_rmsprop()) which adds an exponentially decaying average of squared gradients –> similar effect as momentum.
• Adam (?optimizer_adam()) which is RMSprop + momentum
• Each has their own default learning rates so tune relative to those values
• See https://ruder.io/optimizing-gradient-descent/ for more details

Your turn! (5min)

Retrain our DL model using:

• SGD with momentum (optimizer_sgd())
  • Try default learning rate (0.01) with momentum (0.5, 0.9, 0.99)
  • Try larger learning rates (0.1, 1) with momentum
• RMSprop (optimizer_rmsprop())
  • What is the default learning rate? Train with default.
  • Try a larger learning rate. What happens?
• Adam (optimizer_adam())
  • What is the default learning rate? Train with default.
  • Try a larger learning rate. What happens?

model <- keras_model_sequential() %>%
  layer_dense(units = 512, activation = 'relu', input_shape = n_feat) %>%
  layer_dense(units = 10, activation = 'softmax')

model %>% compile(
  loss = "categorical_crossentropy",
  optimizer = optimizer_rmsprop(learning_rate = 0.1),
  metrics = "accuracy"
  )

b128_rms_0.1_hist <- model %>% fit(
  train_images, train_labels,
  validation_split = 0.2,
  batch_size = 128
  )

plot(b128_rms_0.1_hist)

plot(b128_rms_01_hist) # eta = 0.01

plot(b128_rms_hist) # eta = default = 0.001

Add Callbacks

When training a model, sometimes we want to:

• automatically stop a model once performance has stopped improving
• dynamically adjust values of certain parameters (i.e. learning rate)
• log model information to use or visualize later on
• continually save the model during training and save the model with the best performance

Keras provides a suite of tools called _callbacks that help us to monitor, control, and customize the training procedure.

Stop training at the right time

We often don’t know how many epochs we’ll need to reach a minimum loss. The early stopping callback allows us to automatically stop training after we experience no improvement in our loss after patience number of epochs.

Tip:

• If you are going to use the model after training you always want to retain the “best” model, which is the model with the lowest loss. restore_best_weights will restore the weights for this “best” model even after you’ve passed it by n epochs.

• Sometimes your model will stall on a low validation loss resulting in a tie. Even with early stopping the model will continue to train for all the epochs. min_delta allows you to state some small value that the loss must improve by otherwise it will stop.

model <- keras_model_sequential() %>%
  layer_dense(units = 512, activation = 'relu', input_shape = n_feat) %>%
  layer_dense(units = 10, activation = 'softmax')

model %>% compile(
  loss = "categorical_crossentropy",
  optimizer = optimizer_sgd(lr = 0.1, momentum = 0.9),
  metrics = "accuracy"
)

Warning: the `lr` argument has been renamed to `learning_rate`.

history <- model %>% fit(
  train_images, train_labels,
  validation_split = 0.2,
  batch_size = 128,
  epochs = 20,
  callback = callback_early_stopping(patience = 3,
                                     restore_best_weights = TRUE,
                                     min_delta = 0.0001)
  )

Epoch 1/20

2022-08-04 14:57:56.854099: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:113] Plugin optimizer for device_type GPU is enabled.

  1/375 [..............................] - ETA: 2:33 - loss: 2.3382 - accuracy: 0.1016
  7/375 [..............................] - ETA: 3s - loss: 1.7824 - accuracy: 0.4576  
 12/375 [..............................] - ETA: 4s - loss: 1.3592 - accuracy: 0.5996
 18/375 [>.............................] - ETA: 3s - loss: 1.0865 - accuracy: 0.6793
 25/375 [=>............................] - ETA: 3s - loss: 0.9124 - accuracy: 0.7291
 31/375 [=>............................] - ETA: 3s - loss: 0.8091 - accuracy: 0.7586
 36/375 [=>............................] - ETA: 3s - loss: 0.7729 - accuracy: 0.7700
 41/375 [==>...........................] - ETA: 3s - loss: 0.7387 - accuracy: 0.7799
 47/375 [==>...........................] - ETA: 3s - loss: 0.7043 - accuracy: 0.7904
 53/375 [===>..........................] - ETA: 3s - loss: 0.6721 - accuracy: 0.8007
 59/375 [===>..........................] - ETA: 3s - loss: 0.6481 - accuracy: 0.8064
 65/375 [====>.........................] - ETA: 3s - loss: 0.6202 - accuracy: 0.8145
 71/375 [====>.........................] - ETA: 2s - loss: 0.5950 - accuracy: 0.8212
 76/375 [=====>........................] - ETA: 2s - loss: 0.5796 - accuracy: 0.8257
 81/375 [=====>........................] - ETA: 2s - loss: 0.5627 - accuracy: 0.8305
 87/375 [=====>........................] - ETA: 2s - loss: 0.5463 - accuracy: 0.8350
 92/375 [======>.......................] - ETA: 2s - loss: 0.5319 - accuracy: 0.8394
 98/375 [======>.......................] - ETA: 2s - loss: 0.5197 - accuracy: 0.8430
104/375 [=======>......................] - ETA: 2s - loss: 0.5067 - accuracy: 0.8467
110/375 [=======>......................] - ETA: 2s - loss: 0.4922 - accuracy: 0.8513
116/375 [========>.....................] - ETA: 2s - loss: 0.4800 - accuracy: 0.8549
122/375 [========>.....................] - ETA: 2s - loss: 0.4666 - accuracy: 0.8587
127/375 [=========>....................] - ETA: 2s - loss: 0.4586 - accuracy: 0.8609
131/375 [=========>....................] - ETA: 2s - loss: 0.4525 - accuracy: 0.8628
137/375 [=========>....................] - ETA: 2s - loss: 0.4423 - accuracy: 0.8659
143/375 [==========>...................] - ETA: 2s - loss: 0.4332 - accuracy: 0.8688
149/375 [==========>...................] - ETA: 2s - loss: 0.4230 - accuracy: 0.8723
155/375 [===========>..................] - ETA: 2s - loss: 0.4134 - accuracy: 0.8753
162/375 [===========>..................] - ETA: 2s - loss: 0.4023 - accuracy: 0.8790
169/375 [============>.................] - ETA: 1s - loss: 0.3943 - accuracy: 0.8812
173/375 [============>.................] - ETA: 1s - loss: 0.3898 - accuracy: 0.8826
176/375 [=============>................] - ETA: 1s - loss: 0.3863 - accuracy: 0.8836
183/375 [=============>................] - ETA: 1s - loss: 0.3801 - accuracy: 0.8858
189/375 [==============>...............] - ETA: 1s - loss: 0.3734 - accuracy: 0.8879
195/375 [==============>...............] - ETA: 1s - loss: 0.3673 - accuracy: 0.8897
201/375 [===============>..............] - ETA: 1s - loss: 0.3629 - accuracy: 0.8911
207/375 [===============>..............] - ETA: 1s - loss: 0.3581 - accuracy: 0.8925
214/375 [================>.............] - ETA: 1s - loss: 0.3520 - accuracy: 0.8944
220/375 [================>.............] - ETA: 1s - loss: 0.3474 - accuracy: 0.8958
227/375 [=================>............] - ETA: 1s - loss: 0.3415 - accuracy: 0.8977
233/375 [=================>............] - ETA: 1s - loss: 0.3372 - accuracy: 0.8989
239/375 [==================>...........] - ETA: 1s - loss: 0.3336 - accuracy: 0.8999
245/375 [==================>...........] - ETA: 1s - loss: 0.3294 - accuracy: 0.9011
251/375 [===================>..........] - ETA: 1s - loss: 0.3256 - accuracy: 0.9024
258/375 [===================>..........] - ETA: 1s - loss: 0.3211 - accuracy: 0.9037
265/375 [====================>.........] - ETA: 1s - loss: 0.3165 - accuracy: 0.9049
271/375 [====================>.........] - ETA: 0s - loss: 0.3133 - accuracy: 0.9059
277/375 [=====================>........] - ETA: 0s - loss: 0.3104 - accuracy: 0.9070
283/375 [=====================>........] - ETA: 0s - loss: 0.3070 - accuracy: 0.9079
290/375 [======================>.......] - ETA: 0s - loss: 0.3032 - accuracy: 0.9092
296/375 [======================>.......] - ETA: 0s - loss: 0.3003 - accuracy: 0.9102
303/375 [=======================>......] - ETA: 0s - loss: 0.2964 - accuracy: 0.9113
309/375 [=======================>......] - ETA: 0s - loss: 0.2932 - accuracy: 0.9122
315/375 [========================>.....] - ETA: 0s - loss: 0.2898 - accuracy: 0.9132
322/375 [========================>.....] - ETA: 0s - loss: 0.2863 - accuracy: 0.9143
326/375 [=========================>....] - ETA: 0s - loss: 0.2844 - accuracy: 0.9148
332/375 [=========================>....] - ETA: 0s - loss: 0.2816 - accuracy: 0.9156
338/375 [==========================>...] - ETA: 0s - loss: 0.2787 - accuracy: 0.9165
344/375 [==========================>...] - ETA: 0s - loss: 0.2761 - accuracy: 0.9174
351/375 [===========================>..] - ETA: 0s - loss: 0.2733 - accuracy: 0.9182
358/375 [===========================>..] - ETA: 0s - loss: 0.2703 - accuracy: 0.9191
362/375 [===========================>..] - ETA: 0s - loss: 0.2685 - accuracy: 0.9196
365/375 [============================>.] - ETA: 0s - loss: 0.2678 - accuracy: 0.9198
371/375 [============================>.] - ETA: 0s - loss: 0.2661 - accuracy: 0.9204
375/375 [==============================] - 4s 9ms/step - loss: 0.2643 - accuracy: 0.9209

2022-08-04 14:58:00.740894: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:113] Plugin optimizer for device_type GPU is enabled.


375/375 [==============================] - 5s 13ms/step - loss: 0.2643 - accuracy: 0.9209 - val_loss: 0.1333 - val_accuracy: 0.9603
Epoch 2/20

  1/375 [..............................] - ETA: 2s - loss: 0.2145 - accuracy: 0.9453
  7/375 [..............................] - ETA: 3s - loss: 0.1494 - accuracy: 0.9542
 13/375 [>.............................] - ETA: 3s - loss: 0.1377 - accuracy: 0.9567
 20/375 [>.............................] - ETA: 2s - loss: 0.1237 - accuracy: 0.9617
 26/375 [=>............................] - ETA: 2s - loss: 0.1186 - accuracy: 0.9630
 32/375 [=>............................] - ETA: 2s - loss: 0.1150 - accuracy: 0.9641
 38/375 [==>...........................] - ETA: 2s - loss: 0.1199 - accuracy: 0.9638
 44/375 [==>...........................] - ETA: 2s - loss: 0.1172 - accuracy: 0.9654
 51/375 [===>..........................] - ETA: 2s - loss: 0.1133 - accuracy: 0.9660
 57/375 [===>..........................] - ETA: 2s - loss: 0.1118 - accuracy: 0.9664
 64/375 [====>.........................] - ETA: 2s - loss: 0.1128 - accuracy: 0.9668
 70/375 [====>.........................] - ETA: 2s - loss: 0.1122 - accuracy: 0.9673
 76/375 [=====>........................] - ETA: 2s - loss: 0.1136 - accuracy: 0.9672
 82/375 [=====>........................] - ETA: 2s - loss: 0.1139 - accuracy: 0.9668
 88/375 [======>.......................] - ETA: 2s - loss: 0.1140 - accuracy: 0.9668
 95/375 [======>.......................] - ETA: 2s - loss: 0.1139 - accuracy: 0.9666
102/375 [=======>......................] - ETA: 2s - loss: 0.1116 - accuracy: 0.9670
109/375 [=======>......................] - ETA: 2s - loss: 0.1111 - accuracy: 0.9670
116/375 [========>.....................] - ETA: 2s - loss: 0.1088 - accuracy: 0.9679
122/375 [========>.....................] - ETA: 2s - loss: 0.1081 - accuracy: 0.9682
128/375 [=========>....................] - ETA: 2s - loss: 0.1080 - accuracy: 0.9678
133/375 [=========>....................] - ETA: 2s - loss: 0.1072 - accuracy: 0.9679
140/375 [==========>...................] - ETA: 2s - loss: 0.1076 - accuracy: 0.9679
147/375 [==========>...................] - ETA: 1s - loss: 0.1079 - accuracy: 0.9679
153/375 [===========>..................] - ETA: 1s - loss: 0.1071 - accuracy: 0.9681
159/375 [===========>..................] - ETA: 1s - loss: 0.1063 - accuracy: 0.9683
166/375 [============>.................] - ETA: 1s - loss: 0.1054 - accuracy: 0.9685
172/375 [============>.................] - ETA: 1s - loss: 0.1057 - accuracy: 0.9683
179/375 [=============>................] - ETA: 1s - loss: 0.1056 - accuracy: 0.9683
185/375 [=============>................] - ETA: 1s - loss: 0.1060 - accuracy: 0.9683
192/375 [==============>...............] - ETA: 1s - loss: 0.1061 - accuracy: 0.9683
198/375 [==============>...............] - ETA: 1s - loss: 0.1061 - accuracy: 0.9684
205/375 [===============>..............] - ETA: 1s - loss: 0.1056 - accuracy: 0.9685
212/375 [===============>..............] - ETA: 1s - loss: 0.1053 - accuracy: 0.9686
218/375 [================>.............] - ETA: 1s - loss: 0.1048 - accuracy: 0.9689
225/375 [=================>............] - ETA: 1s - loss: 0.1045 - accuracy: 0.9690
231/375 [=================>............] - ETA: 1s - loss: 0.1044 - accuracy: 0.9692
238/375 [==================>...........] - ETA: 1s - loss: 0.1044 - accuracy: 0.9691
244/375 [==================>...........] - ETA: 1s - loss: 0.1045 - accuracy: 0.9688
250/375 [===================>..........] - ETA: 1s - loss: 0.1042 - accuracy: 0.9688
255/375 [===================>..........] - ETA: 1s - loss: 0.1040 - accuracy: 0.9688
262/375 [===================>..........] - ETA: 0s - loss: 0.1037 - accuracy: 0.9689
268/375 [====================>.........] - ETA: 0s - loss: 0.1033 - accuracy: 0.9689
275/375 [=====================>........] - ETA: 0s - loss: 0.1036 - accuracy: 0.9688
278/375 [=====================>........] - ETA: 0s - loss: 0.1036 - accuracy: 0.9689
284/375 [=====================>........] - ETA: 0s - loss: 0.1034 - accuracy: 0.9690
290/375 [======================>.......] - ETA: 0s - loss: 0.1028 - accuracy: 0.9691
296/375 [======================>.......] - ETA: 0s - loss: 0.1025 - accuracy: 0.9693
303/375 [=======================>......] - ETA: 0s - loss: 0.1022 - accuracy: 0.9694
309/375 [=======================>......] - ETA: 0s - loss: 0.1020 - accuracy: 0.9694
316/375 [========================>.....] - ETA: 0s - loss: 0.1019 - accuracy: 0.9693
323/375 [========================>.....] - ETA: 0s - loss: 0.1015 - accuracy: 0.9693
330/375 [=========================>....] - ETA: 0s - loss: 0.1015 - accuracy: 0.9694
336/375 [=========================>....] - ETA: 0s - loss: 0.1012 - accuracy: 0.9694
342/375 [==========================>...] - ETA: 0s - loss: 0.1009 - accuracy: 0.9696
349/375 [==========================>...] - ETA: 0s - loss: 0.1015 - accuracy: 0.9694
355/375 [===========================>..] - ETA: 0s - loss: 0.1011 - accuracy: 0.9695
362/375 [===========================>..] - ETA: 0s - loss: 0.1009 - accuracy: 0.9697
368/375 [============================>.] - ETA: 0s - loss: 0.1008 - accuracy: 0.9697
375/375 [==============================] - 3s 9ms/step - loss: 0.1003 - accuracy: 0.9699

375/375 [==============================] - 4s 11ms/step - loss: 0.1003 - accuracy: 0.9699 - val_loss: 0.1072 - val_accuracy: 0.9667
Epoch 3/20

  1/375 [..............................] - ETA: 2s - loss: 0.0890 - accuracy: 0.9766
  7/375 [..............................] - ETA: 3s - loss: 0.0843 - accuracy: 0.9743
 14/375 [>.............................] - ETA: 3s - loss: 0.0791 - accuracy: 0.9788
 21/375 [>.............................] - ETA: 2s - loss: 0.0758 - accuracy: 0.9781
 28/375 [=>............................] - ETA: 2s - loss: 0.0722 - accuracy: 0.9802
 33/375 [=>............................] - ETA: 3s - loss: 0.0742 - accuracy: 0.9792
 39/375 [==>...........................] - ETA: 2s - loss: 0.0745 - accuracy: 0.9796
 45/375 [==>...........................] - ETA: 2s - loss: 0.0738 - accuracy: 0.9792
 52/375 [===>..........................] - ETA: 2s - loss: 0.0719 - accuracy: 0.9802
 58/375 [===>..........................] - ETA: 2s - loss: 0.0703 - accuracy: 0.9801
 64/375 [====>.........................] - ETA: 2s - loss: 0.0698 - accuracy: 0.9802
 71/375 [====>.........................] - ETA: 2s - loss: 0.0717 - accuracy: 0.9800
 77/375 [=====>........................] - ETA: 2s - loss: 0.0719 - accuracy: 0.9798
 84/375 [=====>........................] - ETA: 2s - loss: 0.0722 - accuracy: 0.9797
 90/375 [======>.......................] - ETA: 2s - loss: 0.0726 - accuracy: 0.9793
 96/375 [======>.......................] - ETA: 2s - loss: 0.0723 - accuracy: 0.9794
103/375 [=======>......................] - ETA: 2s - loss: 0.0729 - accuracy: 0.9792
110/375 [=======>......................] - ETA: 2s - loss: 0.0730 - accuracy: 0.9793
116/375 [========>.....................] - ETA: 2s - loss: 0.0724 - accuracy: 0.9791
123/375 [========>.....................] - ETA: 2s - loss: 0.0726 - accuracy: 0.9788
129/375 [=========>....................] - ETA: 2s - loss: 0.0724 - accuracy: 0.9788
133/375 [=========>....................] - ETA: 2s - loss: 0.0723 - accuracy: 0.9787
139/375 [==========>...................] - ETA: 2s - loss: 0.0725 - accuracy: 0.9786
145/375 [==========>...................] - ETA: 1s - loss: 0.0735 - accuracy: 0.9781
152/375 [===========>..................] - ETA: 1s - loss: 0.0739 - accuracy: 0.9780
158/375 [===========>..................] - ETA: 1s - loss: 0.0729 - accuracy: 0.9782
164/375 [============>.................] - ETA: 1s - loss: 0.0727 - accuracy: 0.9782
170/375 [============>.................] - ETA: 1s - loss: 0.0729 - accuracy: 0.9781
176/375 [=============>................] - ETA: 1s - loss: 0.0727 - accuracy: 0.9782
182/375 [=============>................] - ETA: 1s - loss: 0.0725 - accuracy: 0.9781
189/375 [==============>...............] - ETA: 1s - loss: 0.0724 - accuracy: 0.9781
196/375 [==============>...............] - ETA: 1s - loss: 0.0724 - accuracy: 0.9780
203/375 [===============>..............] - ETA: 1s - loss: 0.0718 - accuracy: 0.9783
209/375 [===============>..............] - ETA: 1s - loss: 0.0717 - accuracy: 0.9784
215/375 [================>.............] - ETA: 1s - loss: 0.0720 - accuracy: 0.9784
221/375 [================>.............] - ETA: 1s - loss: 0.0718 - accuracy: 0.9783
228/375 [=================>............] - ETA: 1s - loss: 0.0711 - accuracy: 0.9785
234/375 [=================>............] - ETA: 1s - loss: 0.0712 - accuracy: 0.9783
240/375 [==================>...........] - ETA: 1s - loss: 0.0704 - accuracy: 0.9786
247/375 [==================>...........] - ETA: 1s - loss: 0.0703 - accuracy: 0.9787
254/375 [===================>..........] - ETA: 1s - loss: 0.0696 - accuracy: 0.9790
260/375 [===================>..........] - ETA: 0s - loss: 0.0690 - accuracy: 0.9791
267/375 [====================>.........] - ETA: 0s - loss: 0.0701 - accuracy: 0.9789
274/375 [====================>.........] - ETA: 0s - loss: 0.0699 - accuracy: 0.9789
280/375 [=====================>........] - ETA: 0s - loss: 0.0698 - accuracy: 0.9790
286/375 [=====================>........] - ETA: 0s - loss: 0.0692 - accuracy: 0.9792
293/375 [======================>.......] - ETA: 0s - loss: 0.0693 - accuracy: 0.9791
300/375 [=======================>......] - ETA: 0s - loss: 0.0690 - accuracy: 0.9792
307/375 [=======================>......] - ETA: 0s - loss: 0.0691 - accuracy: 0.9792
313/375 [========================>.....] - ETA: 0s - loss: 0.0691 - accuracy: 0.9792
319/375 [========================>.....] - ETA: 0s - loss: 0.0691 - accuracy: 0.9792
325/375 [=========================>....] - ETA: 0s - loss: 0.0688 - accuracy: 0.9794
332/375 [=========================>....] - ETA: 0s - loss: 0.0684 - accuracy: 0.9794
338/375 [==========================>...] - ETA: 0s - loss: 0.0682 - accuracy: 0.9795
345/375 [==========================>...] - ETA: 0s - loss: 0.0678 - accuracy: 0.9796
352/375 [===========================>..] - ETA: 0s - loss: 0.0676 - accuracy: 0.9797
357/375 [===========================>..] - ETA: 0s - loss: 0.0673 - accuracy: 0.9798
363/375 [============================>.] - ETA: 0s - loss: 0.0673 - accuracy: 0.9799
369/375 [============================>.] - ETA: 0s - loss: 0.0671 - accuracy: 0.9799
375/375 [==============================] - 3s 9ms/step - loss: 0.0669 - accuracy: 0.9799

375/375 [==============================] - 4s 11ms/step - loss: 0.0669 - accuracy: 0.9799 - val_loss: 0.0888 - val_accuracy: 0.9725
Epoch 4/20

  1/375 [..............................] - ETA: 2s - loss: 0.0448 - accuracy: 0.9844
  8/375 [..............................] - ETA: 2s - loss: 0.0418 - accuracy: 0.9854
 15/375 [>.............................] - ETA: 2s - loss: 0.0439 - accuracy: 0.9865
 21/375 [>.............................] - ETA: 2s - loss: 0.0448 - accuracy: 0.9862
 28/375 [=>............................] - ETA: 2s - loss: 0.0453 - accuracy: 0.9863
 35/375 [=>............................] - ETA: 2s - loss: 0.0435 - accuracy: 0.9868
 42/375 [==>...........................] - ETA: 2s - loss: 0.0428 - accuracy: 0.9877
 49/375 [==>...........................] - ETA: 2s - loss: 0.0452 - accuracy: 0.9869
 56/375 [===>..........................] - ETA: 2s - loss: 0.0462 - accuracy: 0.9870
 62/375 [===>..........................] - ETA: 2s - loss: 0.0469 - accuracy: 0.9870
 68/375 [====>.........................] - ETA: 2s - loss: 0.0490 - accuracy: 0.9860
 74/375 [====>.........................] - ETA: 2s - loss: 0.0483 - accuracy: 0.9860
 78/375 [=====>........................] - ETA: 2s - loss: 0.0485 - accuracy: 0.9856
 84/375 [=====>........................] - ETA: 2s - loss: 0.0488 - accuracy: 0.9856
 90/375 [======>.......................] - ETA: 2s - loss: 0.0490 - accuracy: 0.9855
 96/375 [======>.......................] - ETA: 2s - loss: 0.0514 - accuracy: 0.9849
102/375 [=======>......................] - ETA: 2s - loss: 0.0512 - accuracy: 0.9850
108/375 [=======>......................] - ETA: 2s - loss: 0.0513 - accuracy: 0.9850
112/375 [=======>......................] - ETA: 2s - loss: 0.0511 - accuracy: 0.9851
118/375 [========>.....................] - ETA: 2s - loss: 0.0506 - accuracy: 0.9852
124/375 [========>.....................] - ETA: 2s - loss: 0.0503 - accuracy: 0.9853
130/375 [=========>....................] - ETA: 2s - loss: 0.0514 - accuracy: 0.9849
136/375 [=========>....................] - ETA: 2s - loss: 0.0517 - accuracy: 0.9849
142/375 [==========>...................] - ETA: 2s - loss: 0.0518 - accuracy: 0.9850
149/375 [==========>...................] - ETA: 2s - loss: 0.0509 - accuracy: 0.9852
155/375 [===========>..................] - ETA: 1s - loss: 0.0503 - accuracy: 0.9854
162/375 [===========>..................] - ETA: 1s - loss: 0.0498 - accuracy: 0.9853
169/375 [============>.................] - ETA: 1s - loss: 0.0494 - accuracy: 0.9854
175/375 [=============>................] - ETA: 1s - loss: 0.0489 - accuracy: 0.9855
181/375 [=============>................] - ETA: 1s - loss: 0.0489 - accuracy: 0.9854
187/375 [=============>................] - ETA: 1s - loss: 0.0499 - accuracy: 0.9850
194/375 [==============>...............] - ETA: 1s - loss: 0.0497 - accuracy: 0.9851
200/375 [===============>..............] - ETA: 1s - loss: 0.0494 - accuracy: 0.9852
207/375 [===============>..............] - ETA: 1s - loss: 0.0490 - accuracy: 0.9854
213/375 [================>.............] - ETA: 1s - loss: 0.0487 - accuracy: 0.9857
220/375 [================>.............] - ETA: 1s - loss: 0.0486 - accuracy: 0.9857
226/375 [=================>............] - ETA: 1s - loss: 0.0481 - accuracy: 0.9859
233/375 [=================>............] - ETA: 1s - loss: 0.0475 - accuracy: 0.9861
239/375 [==================>...........] - ETA: 1s - loss: 0.0476 - accuracy: 0.9861
246/375 [==================>...........] - ETA: 1s - loss: 0.0473 - accuracy: 0.9862
252/375 [===================>..........] - ETA: 1s - loss: 0.0474 - accuracy: 0.9861
258/375 [===================>..........] - ETA: 1s - loss: 0.0475 - accuracy: 0.9859
264/375 [====================>.........] - ETA: 0s - loss: 0.0476 - accuracy: 0.9858
270/375 [====================>.........] - ETA: 0s - loss: 0.0472 - accuracy: 0.9860
276/375 [=====================>........] - ETA: 0s - loss: 0.0470 - accuracy: 0.9860
280/375 [=====================>........] - ETA: 0s - loss: 0.0469 - accuracy: 0.9860
286/375 [=====================>........] - ETA: 0s - loss: 0.0470 - accuracy: 0.9861
292/375 [======================>.......] - ETA: 0s - loss: 0.0468 - accuracy: 0.9862
299/375 [======================>.......] - ETA: 0s - loss: 0.0470 - accuracy: 0.9862
305/375 [=======================>......] - ETA: 0s - loss: 0.0469 - accuracy: 0.9862
311/375 [=======================>......] - ETA: 0s - loss: 0.0469 - accuracy: 0.9863
318/375 [========================>.....] - ETA: 0s - loss: 0.0472 - accuracy: 0.9862
325/375 [=========================>....] - ETA: 0s - loss: 0.0475 - accuracy: 0.9861
331/375 [=========================>....] - ETA: 0s - loss: 0.0472 - accuracy: 0.9861
337/375 [=========================>....] - ETA: 0s - loss: 0.0471 - accuracy: 0.9862
344/375 [==========================>...] - ETA: 0s - loss: 0.0473 - accuracy: 0.9861
351/375 [===========================>..] - ETA: 0s - loss: 0.0476 - accuracy: 0.9860
357/375 [===========================>..] - ETA: 0s - loss: 0.0478 - accuracy: 0.9858
363/375 [============================>.] - ETA: 0s - loss: 0.0478 - accuracy: 0.9857
370/375 [============================>.] - ETA: 0s - loss: 0.0480 - accuracy: 0.9856
375/375 [==============================] - 3s 9ms/step - loss: 0.0479 - accuracy: 0.9856

375/375 [==============================] - 4s 11ms/step - loss: 0.0479 - accuracy: 0.9856 - val_loss: 0.0782 - val_accuracy: 0.9760
Epoch 5/20

  1/375 [..............................] - ETA: 3s - loss: 0.0435 - accuracy: 0.9922
  7/375 [..............................] - ETA: 3s - loss: 0.0319 - accuracy: 0.9911
 13/375 [>.............................] - ETA: 3s - loss: 0.0349 - accuracy: 0.9880
 19/375 [>.............................] - ETA: 3s - loss: 0.0315 - accuracy: 0.9897
 25/375 [=>............................] - ETA: 2s - loss: 0.0304 - accuracy: 0.9906
 31/375 [=>............................] - ETA: 2s - loss: 0.0315 - accuracy: 0.9902
 37/375 [=>............................] - ETA: 2s - loss: 0.0308 - accuracy: 0.9899
 43/375 [==>...........................] - ETA: 2s - loss: 0.0325 - accuracy: 0.9891
 50/375 [===>..........................] - ETA: 2s - loss: 0.0326 - accuracy: 0.9897
 56/375 [===>..........................] - ETA: 2s - loss: 0.0315 - accuracy: 0.9901
 62/375 [===>..........................] - ETA: 2s - loss: 0.0313 - accuracy: 0.9899
 69/375 [====>.........................] - ETA: 2s - loss: 0.0319 - accuracy: 0.9899
 75/375 [=====>........................] - ETA: 2s - loss: 0.0323 - accuracy: 0.9900
 82/375 [=====>........................] - ETA: 2s - loss: 0.0334 - accuracy: 0.9893
 88/375 [======>.......................] - ETA: 2s - loss: 0.0340 - accuracy: 0.9893
 95/375 [======>.......................] - ETA: 2s - loss: 0.0339 - accuracy: 0.9893
102/375 [=======>......................] - ETA: 2s - loss: 0.0334 - accuracy: 0.9894
108/375 [=======>......................] - ETA: 2s - loss: 0.0339 - accuracy: 0.9893
114/375 [========>.....................] - ETA: 2s - loss: 0.0338 - accuracy: 0.9894
120/375 [========>.....................] - ETA: 2s - loss: 0.0344 - accuracy: 0.9889
126/375 [=========>....................] - ETA: 2s - loss: 0.0346 - accuracy: 0.9888
132/375 [=========>....................] - ETA: 2s - loss: 0.0344 - accuracy: 0.9889
139/375 [==========>...................] - ETA: 1s - loss: 0.0345 - accuracy: 0.9888
146/375 [==========>...................] - ETA: 1s - loss: 0.0341 - accuracy: 0.9890
153/375 [===========>..................] - ETA: 1s - loss: 0.0346 - accuracy: 0.9888
159/375 [===========>..................] - ETA: 1s - loss: 0.0348 - accuracy: 0.9887
165/375 [============>.................] - ETA: 1s - loss: 0.0354 - accuracy: 0.9885
172/375 [============>.................] - ETA: 1s - loss: 0.0350 - accuracy: 0.9888
178/375 [=============>................] - ETA: 1s - loss: 0.0347 - accuracy: 0.9889
182/375 [=============>................] - ETA: 1s - loss: 0.0346 - accuracy: 0.9889
188/375 [==============>...............] - ETA: 1s - loss: 0.0348 - accuracy: 0.9889
195/375 [==============>...............] - ETA: 1s - loss: 0.0347 - accuracy: 0.9890
202/375 [===============>..............] - ETA: 1s - loss: 0.0343 - accuracy: 0.9892
208/375 [===============>..............] - ETA: 1s - loss: 0.0340 - accuracy: 0.9892
214/375 [================>.............] - ETA: 1s - loss: 0.0343 - accuracy: 0.9892
221/375 [================>.............] - ETA: 1s - loss: 0.0346 - accuracy: 0.9893
227/375 [=================>............] - ETA: 1s - loss: 0.0350 - accuracy: 0.9890
234/375 [=================>............] - ETA: 1s - loss: 0.0348 - accuracy: 0.9890
241/375 [==================>...........] - ETA: 1s - loss: 0.0346 - accuracy: 0.9891
248/375 [==================>...........] - ETA: 1s - loss: 0.0346 - accuracy: 0.9893
254/375 [===================>..........] - ETA: 1s - loss: 0.0344 - accuracy: 0.9894
260/375 [===================>..........] - ETA: 0s - loss: 0.0343 - accuracy: 0.9894
266/375 [====================>.........] - ETA: 0s - loss: 0.0343 - accuracy: 0.9895
273/375 [====================>.........] - ETA: 0s - loss: 0.0342 - accuracy: 0.9895
279/375 [=====================>........] - ETA: 0s - loss: 0.0342 - accuracy: 0.9895
285/375 [=====================>........] - ETA: 0s - loss: 0.0341 - accuracy: 0.9896
291/375 [======================>.......] - ETA: 0s - loss: 0.0338 - accuracy: 0.9897
297/375 [======================>.......] - ETA: 0s - loss: 0.0337 - accuracy: 0.9898
303/375 [=======================>......] - ETA: 0s - loss: 0.0337 - accuracy: 0.9899
310/375 [=======================>......] - ETA: 0s - loss: 0.0337 - accuracy: 0.9898
315/375 [========================>.....] - ETA: 0s - loss: 0.0336 - accuracy: 0.9899
320/375 [========================>.....] - ETA: 0s - loss: 0.0338 - accuracy: 0.9899
326/375 [=========================>....] - ETA: 0s - loss: 0.0338 - accuracy: 0.9898
332/375 [=========================>....] - ETA: 0s - loss: 0.0336 - accuracy: 0.9899
339/375 [==========================>...] - ETA: 0s - loss: 0.0335 - accuracy: 0.9900
345/375 [==========================>...] - ETA: 0s - loss: 0.0334 - accuracy: 0.9900
351/375 [===========================>..] - ETA: 0s - loss: 0.0336 - accuracy: 0.9899
357/375 [===========================>..] - ETA: 0s - loss: 0.0337 - accuracy: 0.9899
364/375 [============================>.] - ETA: 0s - loss: 0.0335 - accuracy: 0.9900
370/375 [============================>.] - ETA: 0s - loss: 0.0334 - accuracy: 0.9900
375/375 [==============================] - 3s 9ms/step - loss: 0.0335 - accuracy: 0.9900

375/375 [==============================] - 4s 11ms/step - loss: 0.0335 - accuracy: 0.9900 - val_loss: 0.0787 - val_accuracy: 0.9772
Epoch 6/20

  1/375 [..............................] - ETA: 2s - loss: 0.0151 - accuracy: 0.9922
  8/375 [..............................] - ETA: 3s - loss: 0.0194 - accuracy: 0.9922
 14/375 [>.............................] - ETA: 3s - loss: 0.0206 - accuracy: 0.9933
 21/375 [>.............................] - ETA: 2s - loss: 0.0192 - accuracy: 0.9937
 28/375 [=>............................] - ETA: 2s - loss: 0.0202 - accuracy: 0.9939
 35/375 [=>............................] - ETA: 2s - loss: 0.0195 - accuracy: 0.9944
 41/375 [==>...........................] - ETA: 2s - loss: 0.0195 - accuracy: 0.9947
 47/375 [==>...........................] - ETA: 2s - loss: 0.0220 - accuracy: 0.9942
 52/375 [===>..........................] - ETA: 2s - loss: 0.0223 - accuracy: 0.9943
 58/375 [===>..........................] - ETA: 2s - loss: 0.0219 - accuracy: 0.9943
 64/375 [====>.........................] - ETA: 2s - loss: 0.0220 - accuracy: 0.9944
 70/375 [====>.........................] - ETA: 2s - loss: 0.0222 - accuracy: 0.9943
 77/375 [=====>........................] - ETA: 2s - loss: 0.0215 - accuracy: 0.9944
 81/375 [=====>........................] - ETA: 2s - loss: 0.0215 - accuracy: 0.9945
 87/375 [=====>........................] - ETA: 2s - loss: 0.0221 - accuracy: 0.9943
 94/375 [======>.......................] - ETA: 2s - loss: 0.0218 - accuracy: 0.9943
 99/375 [======>.......................] - ETA: 2s - loss: 0.0222 - accuracy: 0.9942
104/375 [=======>......................] - ETA: 2s - loss: 0.0222 - accuracy: 0.9940
110/375 [=======>......................] - ETA: 2s - loss: 0.0221 - accuracy: 0.9939
116/375 [========>.....................] - ETA: 2s - loss: 0.0228 - accuracy: 0.9936
122/375 [========>.....................] - ETA: 2s - loss: 0.0228 - accuracy: 0.9935
128/375 [=========>....................] - ETA: 2s - loss: 0.0232 - accuracy: 0.9934
134/375 [=========>....................] - ETA: 2s - loss: 0.0237 - accuracy: 0.9934
141/375 [==========>...................] - ETA: 2s - loss: 0.0236 - accuracy: 0.9935
145/375 [==========>...................] - ETA: 2s - loss: 0.0239 - accuracy: 0.9933
152/375 [===========>..................] - ETA: 2s - loss: 0.0245 - accuracy: 0.9929
158/375 [===========>..................] - ETA: 2s - loss: 0.0251 - accuracy: 0.9927
164/375 [============>.................] - ETA: 1s - loss: 0.0252 - accuracy: 0.9927
170/375 [============>.................] - ETA: 1s - loss: 0.0252 - accuracy: 0.9926
176/375 [=============>................] - ETA: 1s - loss: 0.0251 - accuracy: 0.9927
182/375 [=============>................] - ETA: 1s - loss: 0.0249 - accuracy: 0.9928
188/375 [==============>...............] - ETA: 1s - loss: 0.0248 - accuracy: 0.9929
194/375 [==============>...............] - ETA: 1s - loss: 0.0249 - accuracy: 0.9929
201/375 [===============>..............] - ETA: 1s - loss: 0.0249 - accuracy: 0.9930
207/375 [===============>..............] - ETA: 1s - loss: 0.0248 - accuracy: 0.9930
214/375 [================>.............] - ETA: 1s - loss: 0.0248 - accuracy: 0.9929
220/375 [================>.............] - ETA: 1s - loss: 0.0248 - accuracy: 0.9929
226/375 [=================>............] - ETA: 1s - loss: 0.0247 - accuracy: 0.9929
232/375 [=================>............] - ETA: 1s - loss: 0.0245 - accuracy: 0.9930
239/375 [==================>...........] - ETA: 1s - loss: 0.0242 - accuracy: 0.9931
245/375 [==================>...........] - ETA: 1s - loss: 0.0240 - accuracy: 0.9932
252/375 [===================>..........] - ETA: 1s - loss: 0.0238 - accuracy: 0.9932
259/375 [===================>..........] - ETA: 1s - loss: 0.0238 - accuracy: 0.9932
265/375 [====================>.........] - ETA: 0s - loss: 0.0238 - accuracy: 0.9933
272/375 [====================>.........] - ETA: 0s - loss: 0.0237 - accuracy: 0.9933
275/375 [=====================>........] - ETA: 0s - loss: 0.0239 - accuracy: 0.9932
279/375 [=====================>........] - ETA: 0s - loss: 0.0240 - accuracy: 0.9932
285/375 [=====================>........] - ETA: 0s - loss: 0.0239 - accuracy: 0.9932
291/375 [======================>.......] - ETA: 0s - loss: 0.0240 - accuracy: 0.9932
297/375 [======================>.......] - ETA: 0s - loss: 0.0241 - accuracy: 0.9931
302/375 [=======================>......] - ETA: 0s - loss: 0.0239 - accuracy: 0.9931
308/375 [=======================>......] - ETA: 0s - loss: 0.0241 - accuracy: 0.9931
314/375 [========================>.....] - ETA: 0s - loss: 0.0239 - accuracy: 0.9931
320/375 [========================>.....] - ETA: 0s - loss: 0.0242 - accuracy: 0.9929
327/375 [=========================>....] - ETA: 0s - loss: 0.0244 - accuracy: 0.9929
333/375 [=========================>....] - ETA: 0s - loss: 0.0245 - accuracy: 0.9929
339/375 [==========================>...] - ETA: 0s - loss: 0.0247 - accuracy: 0.9929
345/375 [==========================>...] - ETA: 0s - loss: 0.0245 - accuracy: 0.9929
351/375 [===========================>..] - ETA: 0s - loss: 0.0244 - accuracy: 0.9930
357/375 [===========================>..] - ETA: 0s - loss: 0.0245 - accuracy: 0.9930
364/375 [============================>.] - ETA: 0s - loss: 0.0244 - accuracy: 0.9929
370/375 [============================>.] - ETA: 0s - loss: 0.0244 - accuracy: 0.9929
375/375 [==============================] - 3s 9ms/step - loss: 0.0244 - accuracy: 0.9929

375/375 [==============================] - 4s 12ms/step - loss: 0.0244 - accuracy: 0.9929 - val_loss: 0.0710 - val_accuracy: 0.9789
Epoch 7/20

  1/375 [..............................] - ETA: 2s - loss: 0.0274 - accuracy: 0.9922
  7/375 [..............................] - ETA: 3s - loss: 0.0142 - accuracy: 0.9967
 14/375 [>.............................] - ETA: 2s - loss: 0.0133 - accuracy: 0.9978
 20/375 [>.............................] - ETA: 2s - loss: 0.0150 - accuracy: 0.9969
 26/375 [=>............................] - ETA: 2s - loss: 0.0140 - accuracy: 0.9973
 33/375 [=>............................] - ETA: 2s - loss: 0.0144 - accuracy: 0.9967
 40/375 [==>...........................] - ETA: 2s - loss: 0.0148 - accuracy: 0.9967
 46/375 [==>...........................] - ETA: 2s - loss: 0.0140 - accuracy: 0.9971
 52/375 [===>..........................] - ETA: 2s - loss: 0.0139 - accuracy: 0.9973
 58/375 [===>..........................] - ETA: 2s - loss: 0.0133 - accuracy: 0.9973
 65/375 [====>.........................] - ETA: 2s - loss: 0.0135 - accuracy: 0.9971
 71/375 [====>.........................] - ETA: 2s - loss: 0.0138 - accuracy: 0.9969
 78/375 [=====>........................] - ETA: 2s - loss: 0.0139 - accuracy: 0.9969
 85/375 [=====>........................] - ETA: 2s - loss: 0.0147 - accuracy: 0.9969
 91/375 [======>.......................] - ETA: 2s - loss: 0.0147 - accuracy: 0.9970
 97/375 [======>.......................] - ETA: 2s - loss: 0.0154 - accuracy: 0.9968
103/375 [=======>......................] - ETA: 2s - loss: 0.0154 - accuracy: 0.9968
109/375 [=======>......................] - ETA: 2s - loss: 0.0153 - accuracy: 0.9969
115/375 [========>.....................] - ETA: 2s - loss: 0.0152 - accuracy: 0.9968
122/375 [========>.....................] - ETA: 2s - loss: 0.0149 - accuracy: 0.9969
129/375 [=========>....................] - ETA: 2s - loss: 0.0146 - accuracy: 0.9970
135/375 [=========>....................] - ETA: 2s - loss: 0.0144 - accuracy: 0.9971
142/375 [==========>...................] - ETA: 2s - loss: 0.0144 - accuracy: 0.9971
149/375 [==========>...................] - ETA: 1s - loss: 0.0143 - accuracy: 0.9971
155/375 [===========>..................] - ETA: 1s - loss: 0.0144 - accuracy: 0.9969
162/375 [===========>..................] - ETA: 1s - loss: 0.0148 - accuracy: 0.9969
168/375 [============>.................] - ETA: 1s - loss: 0.0147 - accuracy: 0.9969
175/375 [=============>................] - ETA: 1s - loss: 0.0148 - accuracy: 0.9969
181/375 [=============>................] - ETA: 1s - loss: 0.0149 - accuracy: 0.9968
187/375 [=============>................] - ETA: 1s - loss: 0.0150 - accuracy: 0.9968
193/375 [==============>...............] - ETA: 1s - loss: 0.0154 - accuracy: 0.9968
199/375 [==============>...............] - ETA: 1s - loss: 0.0155 - accuracy: 0.9967
205/375 [===============>..............] - ETA: 1s - loss: 0.0156 - accuracy: 0.9966
211/375 [===============>..............] - ETA: 1s - loss: 0.0160 - accuracy: 0.9966
217/375 [================>.............] - ETA: 1s - loss: 0.0159 - accuracy: 0.9966
223/375 [================>.............] - ETA: 1s - loss: 0.0159 - accuracy: 0.9965
229/375 [=================>............] - ETA: 1s - loss: 0.0160 - accuracy: 0.9965
235/375 [=================>............] - ETA: 1s - loss: 0.0160 - accuracy: 0.9964
241/375 [==================>...........] - ETA: 1s - loss: 0.0161 - accuracy: 0.9964
248/375 [==================>...........] - ETA: 1s - loss: 0.0161 - accuracy: 0.9965
255/375 [===================>..........] - ETA: 1s - loss: 0.0161 - accuracy: 0.9964
261/375 [===================>..........] - ETA: 0s - loss: 0.0161 - accuracy: 0.9964
268/375 [====================>.........] - ETA: 0s - loss: 0.0161 - accuracy: 0.9964
275/375 [=====================>........] - ETA: 0s - loss: 0.0161 - accuracy: 0.9964
282/375 [=====================>........] - ETA: 0s - loss: 0.0161 - accuracy: 0.9963
289/375 [======================>.......] - ETA: 0s - loss: 0.0162 - accuracy: 0.9962
296/375 [======================>.......] - ETA: 0s - loss: 0.0162 - accuracy: 0.9962
303/375 [=======================>......] - ETA: 0s - loss: 0.0162 - accuracy: 0.9962
309/375 [=======================>......] - ETA: 0s - loss: 0.0161 - accuracy: 0.9962
316/375 [========================>.....] - ETA: 0s - loss: 0.0160 - accuracy: 0.9963
323/375 [========================>.....] - ETA: 0s - loss: 0.0164 - accuracy: 0.9962
330/375 [=========================>....] - ETA: 0s - loss: 0.0163 - accuracy: 0.9961
337/375 [=========================>....] - ETA: 0s - loss: 0.0164 - accuracy: 0.9961
344/375 [==========================>...] - ETA: 0s - loss: 0.0166 - accuracy: 0.9959
351/375 [===========================>..] - ETA: 0s - loss: 0.0167 - accuracy: 0.9959
358/375 [===========================>..] - ETA: 0s - loss: 0.0167 - accuracy: 0.9958
364/375 [============================>.] - ETA: 0s - loss: 0.0168 - accuracy: 0.9958
371/375 [============================>.] - ETA: 0s - loss: 0.0171 - accuracy: 0.9956
375/375 [==============================] - 3s 8ms/step - loss: 0.0171 - accuracy: 0.9956

375/375 [==============================] - 4s 11ms/step - loss: 0.0171 - accuracy: 0.9956 - val_loss: 0.0685 - val_accuracy: 0.9805
Epoch 8/20

  1/375 [..............................] - ETA: 2s - loss: 0.0043 - accuracy: 1.0000
  8/375 [..............................] - ETA: 3s - loss: 0.0104 - accuracy: 0.9990
 15/375 [>.............................] - ETA: 2s - loss: 0.0092 - accuracy: 0.9995
 21/375 [>.............................] - ETA: 2s - loss: 0.0090 - accuracy: 0.9996
 27/375 [=>............................] - ETA: 3s - loss: 0.0090 - accuracy: 0.9994
 34/375 [=>............................] - ETA: 2s - loss: 0.0091 - accuracy: 0.9993
 40/375 [==>...........................] - ETA: 2s - loss: 0.0088 - accuracy: 0.9992
 47/375 [==>...........................] - ETA: 3s - loss: 0.0087 - accuracy: 0.9993
 51/375 [===>..........................] - ETA: 3s - loss: 0.0095 - accuracy: 0.9991
 57/375 [===>..........................] - ETA: 2s - loss: 0.0095 - accuracy: 0.9992
 63/375 [====>.........................] - ETA: 2s - loss: 0.0093 - accuracy: 0.9991
 70/375 [====>.........................] - ETA: 2s - loss: 0.0100 - accuracy: 0.9990
 77/375 [=====>........................] - ETA: 2s - loss: 0.0101 - accuracy: 0.9989
 83/375 [=====>........................] - ETA: 2s - loss: 0.0100 - accuracy: 0.9990
 89/375 [======>.......................] - ETA: 2s - loss: 0.0100 - accuracy: 0.9989
 95/375 [======>.......................] - ETA: 2s - loss: 0.0099 - accuracy: 0.9989
101/375 [=======>......................] - ETA: 2s - loss: 0.0099 - accuracy: 0.9990
107/375 [=======>......................] - ETA: 2s - loss: 0.0098 - accuracy: 0.9990
114/375 [========>.....................] - ETA: 2s - loss: 0.0097 - accuracy: 0.9990
120/375 [========>.....................] - ETA: 2s - loss: 0.0099 - accuracy: 0.9989
126/375 [=========>....................] - ETA: 2s - loss: 0.0100 - accuracy: 0.9988
132/375 [=========>....................] - ETA: 2s - loss: 0.0100 - accuracy: 0.9987
138/375 [==========>...................] - ETA: 2s - loss: 0.0100 - accuracy: 0.9986
144/375 [==========>...................] - ETA: 2s - loss: 0.0099 - accuracy: 0.9986
150/375 [===========>..................] - ETA: 1s - loss: 0.0107 - accuracy: 0.9984
154/375 [===========>..................] - ETA: 1s - loss: 0.0108 - accuracy: 0.9984
161/375 [===========>..................] - ETA: 1s - loss: 0.0107 - accuracy: 0.9984
167/375 [============>.................] - ETA: 1s - loss: 0.0107 - accuracy: 0.9984
174/375 [============>.................] - ETA: 1s - loss: 0.0110 - accuracy: 0.9982
180/375 [=============>................] - ETA: 1s - loss: 0.0109 - accuracy: 0.9983
187/375 [=============>................] - ETA: 1s - loss: 0.0112 - accuracy: 0.9981
194/375 [==============>...............] - ETA: 1s - loss: 0.0112 - accuracy: 0.9981
200/375 [===============>..............] - ETA: 1s - loss: 0.0111 - accuracy: 0.9982
206/375 [===============>..............] - ETA: 1s - loss: 0.0111 - accuracy: 0.9981
213/375 [================>.............] - ETA: 1s - loss: 0.0111 - accuracy: 0.9980
220/375 [================>.............] - ETA: 1s - loss: 0.0110 - accuracy: 0.9980
226/375 [=================>............] - ETA: 1s - loss: 0.0111 - accuracy: 0.9981
232/375 [=================>............] - ETA: 1s - loss: 0.0111 - accuracy: 0.9980
239/375 [==================>...........] - ETA: 1s - loss: 0.0112 - accuracy: 0.9979
246/375 [==================>...........] - ETA: 1s - loss: 0.0111 - accuracy: 0.9980
252/375 [===================>..........] - ETA: 1s - loss: 0.0110 - accuracy: 0.9980
259/375 [===================>..........] - ETA: 1s - loss: 0.0111 - accuracy: 0.9980
265/375 [====================>.........] - ETA: 0s - loss: 0.0110 - accuracy: 0.9980
271/375 [====================>.........] - ETA: 0s - loss: 0.0111 - accuracy: 0.9980
276/375 [=====================>........] - ETA: 0s - loss: 0.0111 - accuracy: 0.9979
282/375 [=====================>........] - ETA: 0s - loss: 0.0111 - accuracy: 0.9979
288/375 [======================>.......] - ETA: 0s - loss: 0.0111 - accuracy: 0.9979
294/375 [======================>.......] - ETA: 0s - loss: 0.0110 - accuracy: 0.9980
300/375 [=======================>......] - ETA: 0s - loss: 0.0110 - accuracy: 0.9980
306/375 [=======================>......] - ETA: 0s - loss: 0.0109 - accuracy: 0.9980
312/375 [=======================>......] - ETA: 0s - loss: 0.0110 - accuracy: 0.9980
319/375 [========================>.....] - ETA: 0s - loss: 0.0109 - accuracy: 0.9980
325/375 [=========================>....] - ETA: 0s - loss: 0.0109 - accuracy: 0.9980
332/375 [=========================>....] - ETA: 0s - loss: 0.0109 - accuracy: 0.9980
338/375 [==========================>...] - ETA: 0s - loss: 0.0111 - accuracy: 0.9979
344/375 [==========================>...] - ETA: 0s - loss: 0.0110 - accuracy: 0.9979
350/375 [===========================>..] - ETA: 0s - loss: 0.0110 - accuracy: 0.9979
356/375 [===========================>..] - ETA: 0s - loss: 0.0111 - accuracy: 0.9979
363/375 [============================>.] - ETA: 0s - loss: 0.0110 - accuracy: 0.9980
370/375 [============================>.] - ETA: 0s - loss: 0.0110 - accuracy: 0.9979
375/375 [==============================] - 3s 9ms/step - loss: 0.0110 - accuracy: 0.9979

375/375 [==============================] - 4s 11ms/step - loss: 0.0110 - accuracy: 0.9979 - val_loss: 0.0686 - val_accuracy: 0.9812
Epoch 9/20

  1/375 [..............................] - ETA: 2s - loss: 0.0036 - accuracy: 1.0000
  8/375 [..............................] - ETA: 2s - loss: 0.0081 - accuracy: 1.0000
 15/375 [>.............................] - ETA: 2s - loss: 0.0070 - accuracy: 1.0000
 21/375 [>.............................] - ETA: 2s - loss: 0.0066 - accuracy: 1.0000
 27/375 [=>............................] - ETA: 3s - loss: 0.0061 - accuracy: 1.0000
 34/375 [=>............................] - ETA: 2s - loss: 0.0063 - accuracy: 0.9998
 40/375 [==>...........................] - ETA: 2s - loss: 0.0062 - accuracy: 0.9996
 47/375 [==>...........................] - ETA: 2s - loss: 0.0066 - accuracy: 0.9995
 54/375 [===>..........................] - ETA: 2s - loss: 0.0066 - accuracy: 0.9994
 61/375 [===>..........................] - ETA: 2s - loss: 0.0068 - accuracy: 0.9994
 68/375 [====>.........................] - ETA: 2s - loss: 0.0067 - accuracy: 0.9994
 75/375 [=====>........................] - ETA: 2s - loss: 0.0071 - accuracy: 0.9992
 82/375 [=====>........................] - ETA: 2s - loss: 0.0080 - accuracy: 0.9990
 88/375 [======>.......................] - ETA: 2s - loss: 0.0079 - accuracy: 0.9990
 90/375 [======>.......................] - ETA: 2s - loss: 0.0078 - accuracy: 0.9990
 97/375 [======>.......................] - ETA: 2s - loss: 0.0076 - accuracy: 0.9991
104/375 [=======>......................] - ETA: 2s - loss: 0.0075 - accuracy: 0.9992
111/375 [=======>......................] - ETA: 2s - loss: 0.0076 - accuracy: 0.9992
118/375 [========>.....................] - ETA: 2s - loss: 0.0076 - accuracy: 0.9992
124/375 [========>.....................] - ETA: 2s - loss: 0.0076 - accuracy: 0.9991
130/375 [=========>....................] - ETA: 2s - loss: 0.0076 - accuracy: 0.9992
136/375 [=========>....................] - ETA: 2s - loss: 0.0076 - accuracy: 0.9991
143/375 [==========>...................] - ETA: 1s - loss: 0.0077 - accuracy: 0.9991
150/375 [===========>..................] - ETA: 1s - loss: 0.0076 - accuracy: 0.9991
156/375 [===========>..................] - ETA: 1s - loss: 0.0077 - accuracy: 0.9991
163/375 [============>.................] - ETA: 1s - loss: 0.0077 - accuracy: 0.9990
170/375 [============>.................] - ETA: 1s - loss: 0.0076 - accuracy: 0.9991
177/375 [=============>................] - ETA: 1s - loss: 0.0076 - accuracy: 0.9991
183/375 [=============>................] - ETA: 1s - loss: 0.0075 - accuracy: 0.9991
190/375 [==============>...............] - ETA: 1s - loss: 0.0077 - accuracy: 0.9991
195/375 [==============>...............] - ETA: 1s - loss: 0.0077 - accuracy: 0.9991
201/375 [===============>..............] - ETA: 1s - loss: 0.0077 - accuracy: 0.9991
208/375 [===============>..............] - ETA: 1s - loss: 0.0077 - accuracy: 0.9991
214/375 [================>.............] - ETA: 1s - loss: 0.0077 - accuracy: 0.9990
221/375 [================>.............] - ETA: 1s - loss: 0.0077 - accuracy: 0.9990
228/375 [=================>............] - ETA: 1s - loss: 0.0077 - accuracy: 0.9989
234/375 [=================>............] - ETA: 1s - loss: 0.0078 - accuracy: 0.9989
240/375 [==================>...........] - ETA: 1s - loss: 0.0079 - accuracy: 0.9989
247/375 [==================>...........] - ETA: 1s - loss: 0.0080 - accuracy: 0.9988
252/375 [===================>..........] - ETA: 1s - loss: 0.0080 - accuracy: 0.9989
258/375 [===================>..........] - ETA: 0s - loss: 0.0081 - accuracy: 0.9988
264/375 [====================>.........] - ETA: 0s - loss: 0.0081 - accuracy: 0.9988
271/375 [====================>.........] - ETA: 0s - loss: 0.0081 - accuracy: 0.9988
277/375 [=====================>........] - ETA: 0s - loss: 0.0081 - accuracy: 0.9988
282/375 [=====================>........] - ETA: 0s - loss: 0.0082 - accuracy: 0.9988
289/375 [======================>.......] - ETA: 0s - loss: 0.0082 - accuracy: 0.9988
296/375 [======================>.......] - ETA: 0s - loss: 0.0082 - accuracy: 0.9988
303/375 [=======================>......] - ETA: 0s - loss: 0.0084 - accuracy: 0.9987
310/375 [=======================>......] - ETA: 0s - loss: 0.0083 - accuracy: 0.9987
316/375 [========================>.....] - ETA: 0s - loss: 0.0083 - accuracy: 0.9987
323/375 [========================>.....] - ETA: 0s - loss: 0.0083 - accuracy: 0.9987
329/375 [=========================>....] - ETA: 0s - loss: 0.0083 - accuracy: 0.9987
335/375 [=========================>....] - ETA: 0s - loss: 0.0083 - accuracy: 0.9987
342/375 [==========================>...] - ETA: 0s - loss: 0.0082 - accuracy: 0.9987
348/375 [==========================>...] - ETA: 0s - loss: 0.0082 - accuracy: 0.9988
355/375 [===========================>..] - ETA: 0s - loss: 0.0082 - accuracy: 0.9987
362/375 [===========================>..] - ETA: 0s - loss: 0.0082 - accuracy: 0.9987
368/375 [============================>.] - ETA: 0s - loss: 0.0082 - accuracy: 0.9987
375/375 [==============================] - 3s 8ms/step - loss: 0.0082 - accuracy: 0.9987

375/375 [==============================] - 4s 11ms/step - loss: 0.0082 - accuracy: 0.9987 - val_loss: 0.0659 - val_accuracy: 0.9820
Epoch 10/20

  1/375 [..............................] - ETA: 3s - loss: 0.0044 - accuracy: 1.0000
  8/375 [..............................] - ETA: 2s - loss: 0.0057 - accuracy: 0.9990
 15/375 [>.............................] - ETA: 2s - loss: 0.0058 - accuracy: 0.9990
 22/375 [>.............................] - ETA: 2s - loss: 0.0062 - accuracy: 0.9989
 29/375 [=>............................] - ETA: 2s - loss: 0.0058 - accuracy: 0.9992
 36/375 [=>............................] - ETA: 2s - loss: 0.0057 - accuracy: 0.9991
 43/375 [==>...........................] - ETA: 2s - loss: 0.0058 - accuracy: 0.9993
 50/375 [===>..........................] - ETA: 2s - loss: 0.0058 - accuracy: 0.9994
 57/375 [===>..........................] - ETA: 2s - loss: 0.0056 - accuracy: 0.9995
 64/375 [====>.........................] - ETA: 2s - loss: 0.0053 - accuracy: 0.9995
 71/375 [====>.........................] - ETA: 2s - loss: 0.0053 - accuracy: 0.9996
 78/375 [=====>........................] - ETA: 2s - loss: 0.0051 - accuracy: 0.9996
 85/375 [=====>........................] - ETA: 2s - loss: 0.0052 - accuracy: 0.9995
 91/375 [======>.......................] - ETA: 2s - loss: 0.0053 - accuracy: 0.9995
 97/375 [======>.......................] - ETA: 2s - loss: 0.0054 - accuracy: 0.9995
104/375 [=======>......................] - ETA: 2s - loss: 0.0054 - accuracy: 0.9995
110/375 [=======>......................] - ETA: 2s - loss: 0.0055 - accuracy: 0.9994
116/375 [========>.....................] - ETA: 2s - loss: 0.0055 - accuracy: 0.9994
122/375 [========>.....................] - ETA: 2s - loss: 0.0057 - accuracy: 0.9994
128/375 [=========>....................] - ETA: 2s - loss: 0.0057 - accuracy: 0.9993
134/375 [=========>....................] - ETA: 1s - loss: 0.0059 - accuracy: 0.9992
140/375 [==========>...................] - ETA: 1s - loss: 0.0063 - accuracy: 0.9992
146/375 [==========>...................] - ETA: 1s - loss: 0.0063 - accuracy: 0.9991
152/375 [===========>..................] - ETA: 1s - loss: 0.0062 - accuracy: 0.9992
158/375 [===========>..................] - ETA: 1s - loss: 0.0062 - accuracy: 0.9992
164/375 [============>.................] - ETA: 1s - loss: 0.0061 - accuracy: 0.9992
170/375 [============>.................] - ETA: 1s - loss: 0.0061 - accuracy: 0.9992
176/375 [=============>................] - ETA: 1s - loss: 0.0060 - accuracy: 0.9992
182/375 [=============>................] - ETA: 1s - loss: 0.0060 - accuracy: 0.9993
188/375 [==============>...............] - ETA: 1s - loss: 0.0059 - accuracy: 0.9993
194/375 [==============>...............] - ETA: 1s - loss: 0.0059 - accuracy: 0.9993
199/375 [==============>...............] - ETA: 1s - loss: 0.0058 - accuracy: 0.9993
206/375 [===============>..............] - ETA: 1s - loss: 0.0059 - accuracy: 0.9993
212/375 [===============>..............] - ETA: 1s - loss: 0.0058 - accuracy: 0.9993
218/375 [================>.............] - ETA: 1s - loss: 0.0058 - accuracy: 0.9994
224/375 [================>.............] - ETA: 1s - loss: 0.0057 - accuracy: 0.9994
230/375 [=================>............] - ETA: 1s - loss: 0.0057 - accuracy: 0.9994
237/375 [=================>............] - ETA: 1s - loss: 0.0056 - accuracy: 0.9994
244/375 [==================>...........] - ETA: 1s - loss: 0.0057 - accuracy: 0.9994
250/375 [===================>..........] - ETA: 1s - loss: 0.0058 - accuracy: 0.9993
256/375 [===================>..........] - ETA: 1s - loss: 0.0058 - accuracy: 0.9994
262/375 [===================>..........] - ETA: 0s - loss: 0.0058 - accuracy: 0.9993
268/375 [====================>.........] - ETA: 0s - loss: 0.0058 - accuracy: 0.9994
275/375 [=====================>........] - ETA: 0s - loss: 0.0058 - accuracy: 0.9993
282/375 [=====================>........] - ETA: 0s - loss: 0.0058 - accuracy: 0.9993
288/375 [======================>.......] - ETA: 0s - loss: 0.0058 - accuracy: 0.9993
295/375 [======================>.......] - ETA: 0s - loss: 0.0057 - accuracy: 0.9994
301/375 [=======================>......] - ETA: 0s - loss: 0.0058 - accuracy: 0.9994
308/375 [=======================>......] - ETA: 0s - loss: 0.0058 - accuracy: 0.9993
315/375 [========================>.....] - ETA: 0s - loss: 0.0058 - accuracy: 0.9993
321/375 [========================>.....] - ETA: 0s - loss: 0.0059 - accuracy: 0.9993
328/375 [=========================>....] - ETA: 0s - loss: 0.0059 - accuracy: 0.9993
335/375 [=========================>....] - ETA: 0s - loss: 0.0059 - accuracy: 0.9993
341/375 [==========================>...] - ETA: 0s - loss: 0.0060 - accuracy: 0.9993
347/375 [==========================>...] - ETA: 0s - loss: 0.0060 - accuracy: 0.9993
354/375 [===========================>..] - ETA: 0s - loss: 0.0060 - accuracy: 0.9993
360/375 [===========================>..] - ETA: 0s - loss: 0.0060 - accuracy: 0.9993
367/375 [============================>.] - ETA: 0s - loss: 0.0060 - accuracy: 0.9993
373/375 [============================>.] - ETA: 0s - loss: 0.0060 - accuracy: 0.9993
375/375 [==============================] - 3s 9ms/step - loss: 0.0060 - accuracy: 0.9993

375/375 [==============================] - 4s 11ms/step - loss: 0.0060 - accuracy: 0.9993 - val_loss: 0.0659 - val_accuracy: 0.9823
Epoch 11/20

  1/375 [..............................] - ETA: 3s - loss: 0.0030 - accuracy: 1.0000
  8/375 [..............................] - ETA: 2s - loss: 0.0035 - accuracy: 1.0000
  9/375 [..............................] - ETA: 4s - loss: 0.0035 - accuracy: 1.0000
 14/375 [>.............................] - ETA: 4s - loss: 0.0034 - accuracy: 1.0000
 21/375 [>.............................] - ETA: 3s - loss: 0.0034 - accuracy: 1.0000
 27/375 [=>............................] - ETA: 3s - loss: 0.0035 - accuracy: 1.0000
 32/375 [=>............................] - ETA: 3s - loss: 0.0033 - accuracy: 1.0000
 39/375 [==>...........................] - ETA: 3s - loss: 0.0032 - accuracy: 1.0000
 46/375 [==>...........................] - ETA: 3s - loss: 0.0034 - accuracy: 1.0000
 52/375 [===>..........................] - ETA: 3s - loss: 0.0035 - accuracy: 1.0000
 58/375 [===>..........................] - ETA: 3s - loss: 0.0035 - accuracy: 1.0000
 65/375 [====>.........................] - ETA: 2s - loss: 0.0035 - accuracy: 1.0000
 71/375 [====>.........................] - ETA: 2s - loss: 0.0036 - accuracy: 1.0000
 78/375 [=====>........................] - ETA: 2s - loss: 0.0035 - accuracy: 1.0000
 84/375 [=====>........................] - ETA: 2s - loss: 0.0035 - accuracy: 1.0000
 90/375 [======>.......................] - ETA: 2s - loss: 0.0035 - accuracy: 1.0000
 97/375 [======>.......................] - ETA: 2s - loss: 0.0036 - accuracy: 1.0000
104/375 [=======>......................] - ETA: 2s - loss: 0.0035 - accuracy: 1.0000
111/375 [=======>......................] - ETA: 2s - loss: 0.0035 - accuracy: 1.0000
118/375 [========>.....................] - ETA: 2s - loss: 0.0035 - accuracy: 1.0000
124/375 [========>.....................] - ETA: 2s - loss: 0.0035 - accuracy: 1.0000
131/375 [=========>....................] - ETA: 2s - loss: 0.0036 - accuracy: 1.0000
137/375 [=========>....................] - ETA: 2s - loss: 0.0036 - accuracy: 1.0000
144/375 [==========>...................] - ETA: 2s - loss: 0.0037 - accuracy: 1.0000
150/375 [===========>..................] - ETA: 1s - loss: 0.0037 - accuracy: 1.0000
156/375 [===========>..................] - ETA: 1s - loss: 0.0038 - accuracy: 0.9999
162/375 [===========>..................] - ETA: 1s - loss: 0.0038 - accuracy: 0.9999
168/375 [============>.................] - ETA: 1s - loss: 0.0038 - accuracy: 0.9999
175/375 [=============>................] - ETA: 1s - loss: 0.0038 - accuracy: 0.9999
181/375 [=============>................] - ETA: 1s - loss: 0.0038 - accuracy: 0.9999
188/375 [==============>...............] - ETA: 1s - loss: 0.0038 - accuracy: 0.9999
194/375 [==============>...............] - ETA: 1s - loss: 0.0038 - accuracy: 0.9999
200/375 [===============>..............] - ETA: 1s - loss: 0.0038 - accuracy: 0.9999
207/375 [===============>..............] - ETA: 1s - loss: 0.0039 - accuracy: 0.9999
213/375 [================>.............] - ETA: 1s - loss: 0.0039 - accuracy: 0.9999
219/375 [================>.............] - ETA: 1s - loss: 0.0039 - accuracy: 0.9999
225/375 [=================>............] - ETA: 1s - loss: 0.0039 - accuracy: 0.9999
231/375 [=================>............] - ETA: 1s - loss: 0.0039 - accuracy: 0.9999
238/375 [==================>...........] - ETA: 1s - loss: 0.0039 - accuracy: 0.9999
243/375 [==================>...........] - ETA: 1s - loss: 0.0039 - accuracy: 0.9999
248/375 [==================>...........] - ETA: 1s - loss: 0.0039 - accuracy: 0.9999
253/375 [===================>..........] - ETA: 1s - loss: 0.0038 - accuracy: 0.9999
258/375 [===================>..........] - ETA: 1s - loss: 0.0038 - accuracy: 0.9999
263/375 [====================>.........] - ETA: 0s - loss: 0.0038 - accuracy: 0.9999
269/375 [====================>.........] - ETA: 0s - loss: 0.0038 - accuracy: 0.9999
275/375 [=====================>........] - ETA: 0s - loss: 0.0038 - accuracy: 0.9999
281/375 [=====================>........] - ETA: 0s - loss: 0.0038 - accuracy: 0.9999
287/375 [=====================>........] - ETA: 0s - loss: 0.0038 - accuracy: 0.9999
294/375 [======================>.......] - ETA: 0s - loss: 0.0038 - accuracy: 0.9999
300/375 [=======================>......] - ETA: 0s - loss: 0.0038 - accuracy: 0.9999
306/375 [=======================>......] - ETA: 0s - loss: 0.0038 - accuracy: 0.9999
312/375 [=======================>......] - ETA: 0s - loss: 0.0038 - accuracy: 0.9999
319/375 [========================>.....] - ETA: 0s - loss: 0.0038 - accuracy: 0.9999
325/375 [=========================>....] - ETA: 0s - loss: 0.0038 - accuracy: 0.9999
332/375 [=========================>....] - ETA: 0s - loss: 0.0038 - accuracy: 0.9999
338/375 [==========================>...] - ETA: 0s - loss: 0.0038 - accuracy: 0.9999
344/375 [==========================>...] - ETA: 0s - loss: 0.0038 - accuracy: 0.9998
350/375 [===========================>..] - ETA: 0s - loss: 0.0038 - accuracy: 0.9998
357/375 [===========================>..] - ETA: 0s - loss: 0.0039 - accuracy: 0.9998
363/375 [============================>.] - ETA: 0s - loss: 0.0039 - accuracy: 0.9998
369/375 [============================>.] - ETA: 0s - loss: 0.0039 - accuracy: 0.9998
375/375 [==============================] - 3s 9ms/step - loss: 0.0039 - accuracy: 0.9998

375/375 [==============================] - 4s 11ms/step - loss: 0.0039 - accuracy: 0.9998 - val_loss: 0.0642 - val_accuracy: 0.9829
Epoch 12/20

  1/375 [..............................] - ETA: 3s - loss: 6.9736e-04 - accuracy: 1.0000
  7/375 [..............................] - ETA: 3s - loss: 0.0027 - accuracy: 1.0000    
 13/375 [>.............................] - ETA: 3s - loss: 0.0027 - accuracy: 1.0000
 20/375 [>.............................] - ETA: 2s - loss: 0.0026 - accuracy: 1.0000
 27/375 [=>............................] - ETA: 2s - loss: 0.0024 - accuracy: 1.0000
 32/375 [=>............................] - ETA: 3s - loss: 0.0023 - accuracy: 1.0000
 37/375 [=>............................] - ETA: 3s - loss: 0.0023 - accuracy: 1.0000
 43/375 [==>...........................] - ETA: 3s - loss: 0.0025 - accuracy: 1.0000
 49/375 [==>...........................] - ETA: 3s - loss: 0.0025 - accuracy: 1.0000
 56/375 [===>..........................] - ETA: 2s - loss: 0.0026 - accuracy: 1.0000
 62/375 [===>..........................] - ETA: 2s - loss: 0.0028 - accuracy: 0.9999
 68/375 [====>.........................] - ETA: 2s - loss: 0.0028 - accuracy: 0.9999
 75/375 [=====>........................] - ETA: 2s - loss: 0.0029 - accuracy: 0.9998
 82/375 [=====>........................] - ETA: 2s - loss: 0.0030 - accuracy: 0.9998
 88/375 [======>.......................] - ETA: 2s - loss: 0.0030 - accuracy: 0.9998
 95/375 [======>.......................] - ETA: 2s - loss: 0.0030 - accuracy: 0.9998
101/375 [=======>......................] - ETA: 2s - loss: 0.0030 - accuracy: 0.9998
107/375 [=======>......................] - ETA: 2s - loss: 0.0030 - accuracy: 0.9999
113/375 [========>.....................] - ETA: 2s - loss: 0.0030 - accuracy: 0.9999
120/375 [========>.....................] - ETA: 2s - loss: 0.0029 - accuracy: 0.9999
127/375 [=========>....................] - ETA: 2s - loss: 0.0030 - accuracy: 0.9999
133/375 [=========>....................] - ETA: 2s - loss: 0.0030 - accuracy: 0.9999
139/375 [==========>...................] - ETA: 2s - loss: 0.0030 - accuracy: 0.9999
145/375 [==========>...................] - ETA: 2s - loss: 0.0030 - accuracy: 0.9998
151/375 [===========>..................] - ETA: 1s - loss: 0.0031 - accuracy: 0.9998
157/375 [===========>..................] - ETA: 1s - loss: 0.0031 - accuracy: 0.9998
163/375 [============>.................] - ETA: 1s - loss: 0.0031 - accuracy: 0.9998
170/375 [============>.................] - ETA: 1s - loss: 0.0032 - accuracy: 0.9998
177/375 [=============>................] - ETA: 1s - loss: 0.0031 - accuracy: 0.9998
183/375 [=============>................] - ETA: 1s - loss: 0.0032 - accuracy: 0.9998
190/375 [==============>...............] - ETA: 1s - loss: 0.0031 - accuracy: 0.9998
197/375 [==============>...............] - ETA: 1s - loss: 0.0031 - accuracy: 0.9998
204/375 [===============>..............] - ETA: 1s - loss: 0.0031 - accuracy: 0.9998
210/375 [===============>..............] - ETA: 1s - loss: 0.0031 - accuracy: 0.9999
216/375 [================>.............] - ETA: 1s - loss: 0.0031 - accuracy: 0.9999
222/375 [================>.............] - ETA: 1s - loss: 0.0031 - accuracy: 0.9999
228/375 [=================>............] - ETA: 1s - loss: 0.0031 - accuracy: 0.9998
234/375 [=================>............] - ETA: 1s - loss: 0.0031 - accuracy: 0.9998
240/375 [==================>...........] - ETA: 1s - loss: 0.0031 - accuracy: 0.9998
246/375 [==================>...........] - ETA: 1s - loss: 0.0031 - accuracy: 0.9998
253/375 [===================>..........] - ETA: 1s - loss: 0.0031 - accuracy: 0.9998
259/375 [===================>..........] - ETA: 1s - loss: 0.0031 - accuracy: 0.9998
266/375 [====================>.........] - ETA: 0s - loss: 0.0031 - accuracy: 0.9999
272/375 [====================>.........] - ETA: 0s - loss: 0.0031 - accuracy: 0.9999
278/375 [=====================>........] - ETA: 0s - loss: 0.0031 - accuracy: 0.9998
284/375 [=====================>........] - ETA: 0s - loss: 0.0031 - accuracy: 0.9998
290/375 [======================>.......] - ETA: 0s - loss: 0.0031 - accuracy: 0.9998
296/375 [======================>.......] - ETA: 0s - loss: 0.0032 - accuracy: 0.9998
302/375 [=======================>......] - ETA: 0s - loss: 0.0032 - accuracy: 0.9998
308/375 [=======================>......] - ETA: 0s - loss: 0.0032 - accuracy: 0.9998
314/375 [========================>.....] - ETA: 0s - loss: 0.0032 - accuracy: 0.9998
320/375 [========================>.....] - ETA: 0s - loss: 0.0032 - accuracy: 0.9998
326/375 [=========================>....] - ETA: 0s - loss: 0.0032 - accuracy: 0.9998
332/375 [=========================>....] - ETA: 0s - loss: 0.0032 - accuracy: 0.9998
337/375 [=========================>....] - ETA: 0s - loss: 0.0032 - accuracy: 0.9998
343/375 [==========================>...] - ETA: 0s - loss: 0.0032 - accuracy: 0.9998
349/375 [==========================>...] - ETA: 0s - loss: 0.0032 - accuracy: 0.9998
355/375 [===========================>..] - ETA: 0s - loss: 0.0032 - accuracy: 0.9998
361/375 [===========================>..] - ETA: 0s - loss: 0.0032 - accuracy: 0.9998
368/375 [============================>.] - ETA: 0s - loss: 0.0032 - accuracy: 0.9998
375/375 [==============================] - 3s 9ms/step - loss: 0.0032 - accuracy: 0.9998

375/375 [==============================] - 4s 11ms/step - loss: 0.0032 - accuracy: 0.9998 - val_loss: 0.0666 - val_accuracy: 0.9819
Epoch 13/20

  1/375 [..............................] - ETA: 3s - loss: 0.0011 - accuracy: 1.0000
  6/375 [..............................] - ETA: 4s - loss: 0.0023 - accuracy: 1.0000
 12/375 [..............................] - ETA: 3s - loss: 0.0027 - accuracy: 1.0000
 18/375 [>.............................] - ETA: 3s - loss: 0.0028 - accuracy: 1.0000
 23/375 [>.............................] - ETA: 3s - loss: 0.0027 - accuracy: 1.0000
 28/375 [=>............................] - ETA: 3s - loss: 0.0026 - accuracy: 1.0000
 34/375 [=>............................] - ETA: 3s - loss: 0.0025 - accuracy: 1.0000
 40/375 [==>...........................] - ETA: 3s - loss: 0.0025 - accuracy: 1.0000
 46/375 [==>...........................] - ETA: 3s - loss: 0.0025 - accuracy: 1.0000
 51/375 [===>..........................] - ETA: 3s - loss: 0.0024 - accuracy: 1.0000
 57/375 [===>..........................] - ETA: 3s - loss: 0.0023 - accuracy: 1.0000
 62/375 [===>..........................] - ETA: 3s - loss: 0.0024 - accuracy: 1.0000
 68/375 [====>.........................] - ETA: 2s - loss: 0.0025 - accuracy: 1.0000
 75/375 [=====>........................] - ETA: 2s - loss: 0.0025 - accuracy: 1.0000
 82/375 [=====>........................] - ETA: 2s - loss: 0.0025 - accuracy: 1.0000
 88/375 [======>.......................] - ETA: 2s - loss: 0.0025 - accuracy: 1.0000
 94/375 [======>.......................] - ETA: 2s - loss: 0.0025 - accuracy: 1.0000
100/375 [=======>......................] - ETA: 2s - loss: 0.0025 - accuracy: 1.0000
106/375 [=======>......................] - ETA: 2s - loss: 0.0025 - accuracy: 1.0000
112/375 [=======>......................] - ETA: 2s - loss: 0.0024 - accuracy: 1.0000
118/375 [========>.....................] - ETA: 2s - loss: 0.0024 - accuracy: 1.0000
124/375 [========>.....................] - ETA: 2s - loss: 0.0024 - accuracy: 1.0000
131/375 [=========>....................] - ETA: 2s - loss: 0.0024 - accuracy: 1.0000
137/375 [=========>....................] - ETA: 2s - loss: 0.0024 - accuracy: 1.0000
143/375 [==========>...................] - ETA: 2s - loss: 0.0024 - accuracy: 1.0000
150/375 [===========>..................] - ETA: 2s - loss: 0.0024 - accuracy: 1.0000
156/375 [===========>..................] - ETA: 2s - loss: 0.0024 - accuracy: 1.0000
162/375 [===========>..................] - ETA: 1s - loss: 0.0024 - accuracy: 1.0000
168/375 [============>.................] - ETA: 1s - loss: 0.0024 - accuracy: 1.0000
174/375 [============>.................] - ETA: 1s - loss: 0.0024 - accuracy: 1.0000
179/375 [=============>................] - ETA: 1s - loss: 0.0023 - accuracy: 1.0000
184/375 [=============>................] - ETA: 1s - loss: 0.0023 - accuracy: 1.0000
189/375 [==============>...............] - ETA: 1s - loss: 0.0023 - accuracy: 1.0000
194/375 [==============>...............] - ETA: 1s - loss: 0.0024 - accuracy: 1.0000
196/375 [==============>...............] - ETA: 1s - loss: 0.0023 - accuracy: 1.0000
202/375 [===============>..............] - ETA: 1s - loss: 0.0024 - accuracy: 1.0000
208/375 [===============>..............] - ETA: 1s - loss: 0.0024 - accuracy: 1.0000
214/375 [================>.............] - ETA: 1s - loss: 0.0024 - accuracy: 1.0000
220/375 [================>.............] - ETA: 1s - loss: 0.0025 - accuracy: 0.9999
226/375 [=================>............] - ETA: 1s - loss: 0.0025 - accuracy: 0.9999
231/375 [=================>............] - ETA: 1s - loss: 0.0024 - accuracy: 0.9999
236/375 [=================>............] - ETA: 1s - loss: 0.0025 - accuracy: 0.9999
241/375 [==================>...........] - ETA: 1s - loss: 0.0025 - accuracy: 0.9999
247/375 [==================>...........] - ETA: 1s - loss: 0.0025 - accuracy: 0.9999
253/375 [===================>..........] - ETA: 1s - loss: 0.0025 - accuracy: 0.9999
259/375 [===================>..........] - ETA: 1s - loss: 0.0025 - accuracy: 0.9999
265/375 [====================>.........] - ETA: 1s - loss: 0.0025 - accuracy: 0.9999
271/375 [====================>.........] - ETA: 0s - loss: 0.0025 - accuracy: 0.9999
278/375 [=====================>........] - ETA: 0s - loss: 0.0025 - accuracy: 0.9999
284/375 [=====================>........] - ETA: 0s - loss: 0.0025 - accuracy: 0.9999
290/375 [======================>.......] - ETA: 0s - loss: 0.0025 - accuracy: 0.9999
296/375 [======================>.......] - ETA: 0s - loss: 0.0025 - accuracy: 0.9999
301/375 [=======================>......] - ETA: 0s - loss: 0.0025 - accuracy: 0.9999
306/375 [=======================>......] - ETA: 0s - loss: 0.0025 - accuracy: 0.9999
311/375 [=======================>......] - ETA: 0s - loss: 0.0025 - accuracy: 0.9999
317/375 [========================>.....] - ETA: 0s - loss: 0.0024 - accuracy: 0.9999
322/375 [========================>.....] - ETA: 0s - loss: 0.0024 - accuracy: 0.9999
328/375 [=========================>....] - ETA: 0s - loss: 0.0025 - accuracy: 0.9999
335/375 [=========================>....] - ETA: 0s - loss: 0.0025 - accuracy: 0.9999
341/375 [==========================>...] - ETA: 0s - loss: 0.0025 - accuracy: 0.9999
347/375 [==========================>...] - ETA: 0s - loss: 0.0025 - accuracy: 0.9999
353/375 [===========================>..] - ETA: 0s - loss: 0.0025 - accuracy: 0.9999
358/375 [===========================>..] - ETA: 0s - loss: 0.0025 - accuracy: 0.9999
363/375 [============================>.] - ETA: 0s - loss: 0.0025 - accuracy: 0.9999
369/375 [============================>.] - ETA: 0s - loss: 0.0025 - accuracy: 0.9999
375/375 [==============================] - 4s 10ms/step - loss: 0.0025 - accuracy: 0.9999

375/375 [==============================] - 5s 12ms/step - loss: 0.0025 - accuracy: 0.9999 - val_loss: 0.0652 - val_accuracy: 0.9826
Epoch 14/20

  1/375 [..............................] - ETA: 3s - loss: 6.4595e-04 - accuracy: 1.0000
  7/375 [..............................] - ETA: 3s - loss: 0.0016 - accuracy: 1.0000    
 13/375 [>.............................] - ETA: 3s - loss: 0.0015 - accuracy: 1.0000
 19/375 [>.............................] - ETA: 3s - loss: 0.0017 - accuracy: 1.0000
 25/375 [=>............................] - ETA: 3s - loss: 0.0019 - accuracy: 1.0000
 32/375 [=>............................] - ETA: 3s - loss: 0.0019 - accuracy: 1.0000
 38/375 [==>...........................] - ETA: 2s - loss: 0.0020 - accuracy: 1.0000
 43/375 [==>...........................] - ETA: 3s - loss: 0.0020 - accuracy: 1.0000
 49/375 [==>...........................] - ETA: 2s - loss: 0.0020 - accuracy: 1.0000
 55/375 [===>..........................] - ETA: 2s - loss: 0.0020 - accuracy: 1.0000
 61/375 [===>..........................] - ETA: 2s - loss: 0.0020 - accuracy: 1.0000
 67/375 [====>.........................] - ETA: 2s - loss: 0.0019 - accuracy: 1.0000
 73/375 [====>.........................] - ETA: 2s - loss: 0.0019 - accuracy: 1.0000
 80/375 [=====>........................] - ETA: 2s - loss: 0.0020 - accuracy: 1.0000
 87/375 [=====>........................] - ETA: 2s - loss: 0.0020 - accuracy: 1.0000
 93/375 [======>.......................] - ETA: 2s - loss: 0.0020 - accuracy: 1.0000
100/375 [=======>......................] - ETA: 2s - loss: 0.0020 - accuracy: 1.0000
107/375 [=======>......................] - ETA: 2s - loss: 0.0020 - accuracy: 1.0000
113/375 [========>.....................] - ETA: 2s - loss: 0.0020 - accuracy: 1.0000
119/375 [========>.....................] - ETA: 2s - loss: 0.0020 - accuracy: 1.0000
126/375 [=========>....................] - ETA: 2s - loss: 0.0020 - accuracy: 1.0000
131/375 [=========>....................] - ETA: 2s - loss: 0.0020 - accuracy: 1.0000
137/375 [=========>....................] - ETA: 2s - loss: 0.0020 - accuracy: 1.0000
144/375 [==========>...................] - ETA: 2s - loss: 0.0020 - accuracy: 1.0000
150/375 [===========>..................] - ETA: 2s - loss: 0.0020 - accuracy: 1.0000
152/375 [===========>..................] - ETA: 2s - loss: 0.0020 - accuracy: 1.0000
158/375 [===========>..................] - ETA: 2s - loss: 0.0020 - accuracy: 1.0000
164/375 [============>.................] - ETA: 1s - loss: 0.0020 - accuracy: 1.0000
170/375 [============>.................] - ETA: 1s - loss: 0.0020 - accuracy: 1.0000
174/375 [============>.................] - ETA: 1s - loss: 0.0020 - accuracy: 1.0000
180/375 [=============>................] - ETA: 1s - loss: 0.0020 - accuracy: 1.0000
186/375 [=============>................] - ETA: 1s - loss: 0.0020 - accuracy: 1.0000
192/375 [==============>...............] - ETA: 1s - loss: 0.0020 - accuracy: 1.0000
197/375 [==============>...............] - ETA: 1s - loss: 0.0020 - accuracy: 1.0000
202/375 [===============>..............] - ETA: 1s - loss: 0.0020 - accuracy: 1.0000
208/375 [===============>..............] - ETA: 1s - loss: 0.0020 - accuracy: 1.0000
214/375 [================>.............] - ETA: 1s - loss: 0.0020 - accuracy: 1.0000
220/375 [================>.............] - ETA: 1s - loss: 0.0020 - accuracy: 1.0000
226/375 [=================>............] - ETA: 1s - loss: 0.0020 - accuracy: 1.0000
231/375 [=================>............] - ETA: 1s - loss: 0.0020 - accuracy: 1.0000
237/375 [=================>............] - ETA: 1s - loss: 0.0020 - accuracy: 1.0000
243/375 [==================>...........] - ETA: 1s - loss: 0.0020 - accuracy: 1.0000
250/375 [===================>..........] - ETA: 1s - loss: 0.0020 - accuracy: 1.0000
257/375 [===================>..........] - ETA: 1s - loss: 0.0020 - accuracy: 1.0000
264/375 [====================>.........] - ETA: 1s - loss: 0.0020 - accuracy: 1.0000
270/375 [====================>.........] - ETA: 0s - loss: 0.0020 - accuracy: 1.0000
277/375 [=====================>........] - ETA: 0s - loss: 0.0020 - accuracy: 1.0000
283/375 [=====================>........] - ETA: 0s - loss: 0.0020 - accuracy: 1.0000
289/375 [======================>.......] - ETA: 0s - loss: 0.0020 - accuracy: 1.0000
295/375 [======================>.......] - ETA: 0s - loss: 0.0020 - accuracy: 1.0000
301/375 [=======================>......] - ETA: 0s - loss: 0.0020 - accuracy: 1.0000
307/375 [=======================>......] - ETA: 0s - loss: 0.0020 - accuracy: 1.0000
313/375 [========================>.....] - ETA: 0s - loss: 0.0020 - accuracy: 1.0000
319/375 [========================>.....] - ETA: 0s - loss: 0.0020 - accuracy: 1.0000
325/375 [=========================>....] - ETA: 0s - loss: 0.0020 - accuracy: 1.0000
331/375 [=========================>....] - ETA: 0s - loss: 0.0020 - accuracy: 1.0000
337/375 [=========================>....] - ETA: 0s - loss: 0.0020 - accuracy: 1.0000
343/375 [==========================>...] - ETA: 0s - loss: 0.0020 - accuracy: 1.0000
349/375 [==========================>...] - ETA: 0s - loss: 0.0020 - accuracy: 1.0000
355/375 [===========================>..] - ETA: 0s - loss: 0.0020 - accuracy: 1.0000
361/375 [===========================>..] - ETA: 0s - loss: 0.0020 - accuracy: 1.0000
368/375 [============================>.] - ETA: 0s - loss: 0.0020 - accuracy: 1.0000
374/375 [============================>.] - ETA: 0s - loss: 0.0020 - accuracy: 1.0000
375/375 [==============================] - 3s 9ms/step - loss: 0.0020 - accuracy: 1.0000

375/375 [==============================] - 4s 12ms/step - loss: 0.0020 - accuracy: 1.0000 - val_loss: 0.0665 - val_accuracy: 0.9827

history

Final epoch (plot to see history):
        loss: 0.002032
    accuracy: 1
    val_loss: 0.06652
val_accuracy: 0.9827 

Note how we did not train for all the epochs…only until our loss didn’t improve by 0.0001 for 3 consistent epochs.

plot(history)

Add a learning rate scheduler

Although adaptive learning rate optimizers adjust the velocity of weight updates depending on the loss surface, we can also incorporate additional ways to modify the learning rate “on the fly”.

Options:

• Learning rate decay reduces the learning rate at each epoch. Note the decay argument in ?optimizer_xxx().

• There has been some great research on cyclical learning rates (see https://arxiv.org/abs/1506.01186). You can incorporate custom learning rates such as this with callback_learning_rate_scheduler(). This is more advanced but definitely worth reading up on post-workshop.

• A simpler, and very practical approach, is to reduce the learning rate after the model has stopped improving with callback_reduce_lr_on_plateau(). This approach allows the model to tell us when to reduce the learning rate.

Tip: When using decay or callback_reduce_lr_on_plateau(), you can usually increase the base learning rate so that you learn quickly early on and then the learning rate will reduce so you can take smaller steps as you approach the minimum loss.

model <- keras_model_sequential() %>%
  layer_dense(units = 512, activation = 'relu', input_shape = n_feat) %>%
  layer_dense(units = 10, activation = 'softmax')

summary(model)

Model: "sequential_10"
__________________________________________________________________________________________________________
 Layer (type)                                  Output Shape                               Param #         
==========================================================================================================
 dense_28 (Dense)                              (None, 512)                                401920          
 dense_27 (Dense)                              (None, 10)                                 5130            
==========================================================================================================
Total params: 407,050
Trainable params: 407,050
Non-trainable params: 0
__________________________________________________________________________________________________________

model %>% compile(
  loss = "categorical_crossentropy",
  optimizer = optimizer_sgd(learning_rate = 0.1, momentum = 0.9),
  metrics = "accuracy"
)

history <- model %>% fit(
  train_images, train_labels,
  validation_split = 0.2,
  batch_size = 128,
  epochs = 20,
  callback = list(
    # callback_early_stopping(patience = 3, restore_best_weights = TRUE, min_delta = 0.0001),
    callback_reduce_lr_on_plateau(patience = 1, factor = 0.1)
    )
  )

history

Final epoch (plot to see history):
        loss: 0.007527
    accuracy: 0.9993
    val_loss: 0.0628
val_accuracy: 0.9821
          lr: 0.00000000001 

plot(history)

plot(history$metrics$lr)

Explore Model Capacity

Now that we have found a pretty good learning rate and we have good control over our training procedure with callbacks, we can start to assess how the capacity of our model effects performance.

Recall that we control capacity two ways:

• width: the number of units in a hidden layer
• depth: the number of hidden layers

Your turn! (5min)

Explore different model capacities while all other parameters constant. Try increasing

• Width: remember that the number of units is usually a power of 2 (i.e. 32, 64, 128, 256, 512, 1024).
• Depth: try using 2 or 3 hidden layers

summary(model_2_layer_v1)

Model: "sequential_7"
___________________________________________________________________________________________________________
 Layer (type)                                   Output Shape                              Param #          
===========================================================================================================
 dense_18 (Dense)                               (None, 128)                               100480           
 dense_17 (Dense)                               (None, 64)                                8256             
 dense_16 (Dense)                               (None, 10)                                650              
===========================================================================================================
Total params: 109,386
Trainable params: 109,386
Non-trainable params: 0
___________________________________________________________________________________________________________

model_2_layer_v1 %>% compile(
  loss = "categorical_crossentropy",
  optimizer = optimizer_sgd(lr = 0.1, momentum = 0.9),
  metrics = "accuracy"
)

Warning: the `lr` argument has been renamed to `learning_rate`.

history <- model_2_layer_v1 %>% fit(
  train_images, train_labels,
  validation_split = 0.2,
  batch_size = 128,
  epochs = 20,
  callback = list(
    # callback_early_stopping(patience = 3, restore_best_weights = TRUE, min_delta = 0.0001),
    callback_reduce_lr_on_plateau(patience = 1, factor = 0.1)
    )
  )

Epoch 1/20

2022-08-04 15:37:49.905746: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:113] Plugin optimizer for device_type GPU is enabled.

  1/375 [..............................] - ETA: 3:16 - loss: 2.3597 - accuracy: 0.0703
  7/375 [..............................] - ETA: 3s - loss: 2.0831 - accuracy: 0.3114  
 13/375 [>.............................] - ETA: 3s - loss: 1.6223 - accuracy: 0.4904
 19/375 [>.............................] - ETA: 3s - loss: 1.3448 - accuracy: 0.5781
 24/375 [>.............................] - ETA: 3s - loss: 1.2167 - accuracy: 0.6201
 29/375 [=>............................] - ETA: 3s - loss: 1.1378 - accuracy: 0.6479
 36/375 [=>............................] - ETA: 3s - loss: 1.0424 - accuracy: 0.6758
 41/375 [==>...........................] - ETA: 3s - loss: 0.9685 - accuracy: 0.6989
 47/375 [==>...........................] - ETA: 3s - loss: 0.8982 - accuracy: 0.7224
 53/375 [===>..........................] - ETA: 3s - loss: 0.8431 - accuracy: 0.7410
 59/375 [===>..........................] - ETA: 2s - loss: 0.7963 - accuracy: 0.7544
 64/375 [====>.........................] - ETA: 2s - loss: 0.7556 - accuracy: 0.7659
 69/375 [====>.........................] - ETA: 2s - loss: 0.7223 - accuracy: 0.7760
 75/375 [=====>........................] - ETA: 2s - loss: 0.6916 - accuracy: 0.7862
 81/375 [=====>........................] - ETA: 2s - loss: 0.6610 - accuracy: 0.7954
 87/375 [=====>........................] - ETA: 2s - loss: 0.6419 - accuracy: 0.8018
 92/375 [======>.......................] - ETA: 2s - loss: 0.6227 - accuracy: 0.8078
 95/375 [======>.......................] - ETA: 2s - loss: 0.6114 - accuracy: 0.8116
100/375 [=======>......................] - ETA: 2s - loss: 0.5968 - accuracy: 0.8160
103/375 [=======>......................] - ETA: 2s - loss: 0.5869 - accuracy: 0.8186
108/375 [=======>......................] - ETA: 2s - loss: 0.5697 - accuracy: 0.8241
113/375 [========>.....................] - ETA: 2s - loss: 0.5547 - accuracy: 0.8289
118/375 [========>.....................] - ETA: 2s - loss: 0.5421 - accuracy: 0.8331
122/375 [========>.....................] - ETA: 2s - loss: 0.5324 - accuracy: 0.8361
128/375 [=========>....................] - ETA: 2s - loss: 0.5170 - accuracy: 0.8410
134/375 [=========>....................] - ETA: 2s - loss: 0.5034 - accuracy: 0.8450
140/375 [==========>...................] - ETA: 2s - loss: 0.4921 - accuracy: 0.8486
147/375 [==========>...................] - ETA: 2s - loss: 0.4792 - accuracy: 0.8525
153/375 [===========>..................] - ETA: 2s - loss: 0.4675 - accuracy: 0.8561
159/375 [===========>..................] - ETA: 2s - loss: 0.4578 - accuracy: 0.8588
163/375 [============>.................] - ETA: 2s - loss: 0.4520 - accuracy: 0.8606
169/375 [============>.................] - ETA: 2s - loss: 0.4431 - accuracy: 0.8639
175/375 [=============>................] - ETA: 1s - loss: 0.4350 - accuracy: 0.8662
181/375 [=============>................] - ETA: 1s - loss: 0.4287 - accuracy: 0.8682
187/375 [=============>................] - ETA: 1s - loss: 0.4209 - accuracy: 0.8707
193/375 [==============>...............] - ETA: 1s - loss: 0.4138 - accuracy: 0.8730
196/375 [==============>...............] - ETA: 1s - loss: 0.4107 - accuracy: 0.8740
202/375 [===============>..............] - ETA: 1s - loss: 0.4049 - accuracy: 0.8757
208/375 [===============>..............] - ETA: 1s - loss: 0.3985 - accuracy: 0.8777
214/375 [================>.............] - ETA: 1s - loss: 0.3945 - accuracy: 0.8789
221/375 [================>.............] - ETA: 1s - loss: 0.3887 - accuracy: 0.8808
227/375 [=================>............] - ETA: 1s - loss: 0.3833 - accuracy: 0.8824
233/375 [=================>............] - ETA: 1s - loss: 0.3788 - accuracy: 0.8837
239/375 [==================>...........] - ETA: 1s - loss: 0.3738 - accuracy: 0.8855
246/375 [==================>...........] - ETA: 1s - loss: 0.3671 - accuracy: 0.8874
252/375 [===================>..........] - ETA: 1s - loss: 0.3617 - accuracy: 0.8891
258/375 [===================>..........] - ETA: 1s - loss: 0.3570 - accuracy: 0.8905
264/375 [====================>.........] - ETA: 1s - loss: 0.3533 - accuracy: 0.8917
271/375 [====================>.........] - ETA: 0s - loss: 0.3488 - accuracy: 0.8932
277/375 [=====================>........] - ETA: 0s - loss: 0.3449 - accuracy: 0.8944
283/375 [=====================>........] - ETA: 0s - loss: 0.3411 - accuracy: 0.8955
289/375 [======================>.......] - ETA: 0s - loss: 0.3379 - accuracy: 0.8964
295/375 [======================>.......] - ETA: 0s - loss: 0.3341 - accuracy: 0.8975
301/375 [=======================>......] - ETA: 0s - loss: 0.3316 - accuracy: 0.8982
307/375 [=======================>......] - ETA: 0s - loss: 0.3283 - accuracy: 0.8994
313/375 [========================>.....] - ETA: 0s - loss: 0.3245 - accuracy: 0.9004
318/375 [========================>.....] - ETA: 0s - loss: 0.3222 - accuracy: 0.9010
324/375 [========================>.....] - ETA: 0s - loss: 0.3195 - accuracy: 0.9018
330/375 [=========================>....] - ETA: 0s - loss: 0.3163 - accuracy: 0.9028
336/375 [=========================>....] - ETA: 0s - loss: 0.3134 - accuracy: 0.9039
342/375 [==========================>...] - ETA: 0s - loss: 0.3110 - accuracy: 0.9047
348/375 [==========================>...] - ETA: 0s - loss: 0.3089 - accuracy: 0.9052
354/375 [===========================>..] - ETA: 0s - loss: 0.3069 - accuracy: 0.9057
360/375 [===========================>..] - ETA: 0s - loss: 0.3044 - accuracy: 0.9065
366/375 [============================>.] - ETA: 0s - loss: 0.3018 - accuracy: 0.9073
372/375 [============================>.] - ETA: 0s - loss: 0.3003 - accuracy: 0.9078
375/375 [==============================] - 4s 9ms/step - loss: 0.2992 - accuracy: 0.9081

2022-08-04 15:37:53.911289: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:113] Plugin optimizer for device_type GPU is enabled.


375/375 [==============================] - 6s 13ms/step - loss: 0.2992 - accuracy: 0.9081 - val_loss: 0.1599 - val_accuracy: 0.9498 - lr: 0.1000
Epoch 2/20

  1/375 [..............................] - ETA: 2s - loss: 0.0654 - accuracy: 0.9922
  7/375 [..............................] - ETA: 3s - loss: 0.1268 - accuracy: 0.9598
 13/375 [>.............................] - ETA: 3s - loss: 0.1171 - accuracy: 0.9639
 19/375 [>.............................] - ETA: 3s - loss: 0.1181 - accuracy: 0.9646
 25/375 [=>............................] - ETA: 3s - loss: 0.1204 - accuracy: 0.9644
 32/375 [=>............................] - ETA: 2s - loss: 0.1149 - accuracy: 0.9663
 39/375 [==>...........................] - ETA: 2s - loss: 0.1174 - accuracy: 0.9645
 46/375 [==>...........................] - ETA: 2s - loss: 0.1171 - accuracy: 0.9645
 53/375 [===>..........................] - ETA: 2s - loss: 0.1180 - accuracy: 0.9636
 60/375 [===>..........................] - ETA: 2s - loss: 0.1144 - accuracy: 0.9643
 67/375 [====>.........................] - ETA: 2s - loss: 0.1132 - accuracy: 0.9649
 74/375 [====>.........................] - ETA: 2s - loss: 0.1150 - accuracy: 0.9641
 81/375 [=====>........................] - ETA: 2s - loss: 0.1151 - accuracy: 0.9641
 88/375 [======>.......................] - ETA: 2s - loss: 0.1156 - accuracy: 0.9641
 94/375 [======>.......................] - ETA: 2s - loss: 0.1175 - accuracy: 0.9634
100/375 [=======>......................] - ETA: 2s - loss: 0.1163 - accuracy: 0.9638
106/375 [=======>......................] - ETA: 2s - loss: 0.1159 - accuracy: 0.9641
112/375 [=======>......................] - ETA: 2s - loss: 0.1171 - accuracy: 0.9638
118/375 [========>.....................] - ETA: 2s - loss: 0.1169 - accuracy: 0.9639
125/375 [=========>....................] - ETA: 2s - loss: 0.1171 - accuracy: 0.9638
132/375 [=========>....................] - ETA: 1s - loss: 0.1163 - accuracy: 0.9641
139/375 [==========>...................] - ETA: 1s - loss: 0.1178 - accuracy: 0.9638
146/375 [==========>...................] - ETA: 1s - loss: 0.1178 - accuracy: 0.9638
153/375 [===========>..................] - ETA: 1s - loss: 0.1180 - accuracy: 0.9636
160/375 [===========>..................] - ETA: 1s - loss: 0.1179 - accuracy: 0.9636
168/375 [============>.................] - ETA: 1s - loss: 0.1176 - accuracy: 0.9635
175/375 [=============>................] - ETA: 1s - loss: 0.1192 - accuracy: 0.9630
181/375 [=============>................] - ETA: 1s - loss: 0.1188 - accuracy: 0.9631
187/375 [=============>................] - ETA: 1s - loss: 0.1180 - accuracy: 0.9635
194/375 [==============>...............] - ETA: 1s - loss: 0.1166 - accuracy: 0.9638
200/375 [===============>..............] - ETA: 1s - loss: 0.1163 - accuracy: 0.9640
207/375 [===============>..............] - ETA: 1s - loss: 0.1170 - accuracy: 0.9638
213/375 [================>.............] - ETA: 1s - loss: 0.1160 - accuracy: 0.9640
218/375 [================>.............] - ETA: 1s - loss: 0.1165 - accuracy: 0.9640
225/375 [=================>............] - ETA: 1s - loss: 0.1163 - accuracy: 0.9641
232/375 [=================>............] - ETA: 1s - loss: 0.1157 - accuracy: 0.9643
238/375 [==================>...........] - ETA: 1s - loss: 0.1167 - accuracy: 0.9643
245/375 [==================>...........] - ETA: 1s - loss: 0.1168 - accuracy: 0.9641
252/375 [===================>..........] - ETA: 1s - loss: 0.1160 - accuracy: 0.9644
259/375 [===================>..........] - ETA: 0s - loss: 0.1167 - accuracy: 0.9644
266/375 [====================>.........] - ETA: 0s - loss: 0.1168 - accuracy: 0.9643
273/375 [====================>.........] - ETA: 0s - loss: 0.1161 - accuracy: 0.9646
279/375 [=====================>........] - ETA: 0s - loss: 0.1156 - accuracy: 0.9647
285/375 [=====================>........] - ETA: 0s - loss: 0.1150 - accuracy: 0.9649
291/375 [======================>.......] - ETA: 0s - loss: 0.1148 - accuracy: 0.9649
297/375 [======================>.......] - ETA: 0s - loss: 0.1144 - accuracy: 0.9649
303/375 [=======================>......] - ETA: 0s - loss: 0.1140 - accuracy: 0.9651
309/375 [=======================>......] - ETA: 0s - loss: 0.1145 - accuracy: 0.9649
315/375 [========================>.....] - ETA: 0s - loss: 0.1147 - accuracy: 0.9649
322/375 [========================>.....] - ETA: 0s - loss: 0.1150 - accuracy: 0.9648
329/375 [=========================>....] - ETA: 0s - loss: 0.1149 - accuracy: 0.9648
336/375 [=========================>....] - ETA: 0s - loss: 0.1142 - accuracy: 0.9649
342/375 [==========================>...] - ETA: 0s - loss: 0.1142 - accuracy: 0.9650
347/375 [==========================>...] - ETA: 0s - loss: 0.1141 - accuracy: 0.9650
353/375 [===========================>..] - ETA: 0s - loss: 0.1143 - accuracy: 0.9651
359/375 [===========================>..] - ETA: 0s - loss: 0.1141 - accuracy: 0.9652
365/375 [============================>.] - ETA: 0s - loss: 0.1141 - accuracy: 0.9651
372/375 [============================>.] - ETA: 0s - loss: 0.1138 - accuracy: 0.9652
375/375 [==============================] - 3s 8ms/step - loss: 0.1139 - accuracy: 0.9653

375/375 [==============================] - 4s 11ms/step - loss: 0.1139 - accuracy: 0.9653 - val_loss: 0.1082 - val_accuracy: 0.9663 - lr: 0.1000
Epoch 3/20

  1/375 [..............................] - ETA: 4s - loss: 0.0787 - accuracy: 0.9766
  6/375 [..............................] - ETA: 3s - loss: 0.0656 - accuracy: 0.9740
 13/375 [>.............................] - ETA: 3s - loss: 0.0795 - accuracy: 0.9730
 20/375 [>.............................] - ETA: 3s - loss: 0.0794 - accuracy: 0.9719
 27/375 [=>............................] - ETA: 2s - loss: 0.0811 - accuracy: 0.9716
 34/375 [=>............................] - ETA: 2s - loss: 0.0745 - accuracy: 0.9745
 40/375 [==>...........................] - ETA: 2s - loss: 0.0741 - accuracy: 0.9746
 46/375 [==>...........................] - ETA: 2s - loss: 0.0750 - accuracy: 0.9738
 52/375 [===>..........................] - ETA: 2s - loss: 0.0759 - accuracy: 0.9745
 58/375 [===>..........................] - ETA: 2s - loss: 0.0744 - accuracy: 0.9754
 63/375 [====>.........................] - ETA: 2s - loss: 0.0740 - accuracy: 0.9758
 69/375 [====>.........................] - ETA: 3s - loss: 0.0729 - accuracy: 0.9764
 73/375 [====>.........................] - ETA: 3s - loss: 0.0741 - accuracy: 0.9759
 79/375 [=====>........................] - ETA: 2s - loss: 0.0746 - accuracy: 0.9756
 85/375 [=====>........................] - ETA: 2s - loss: 0.0777 - accuracy: 0.9744
 90/375 [======>.......................] - ETA: 2s - loss: 0.0775 - accuracy: 0.9748
 96/375 [======>.......................] - ETA: 2s - loss: 0.0770 - accuracy: 0.9749
101/375 [=======>......................] - ETA: 2s - loss: 0.0774 - accuracy: 0.9748
107/375 [=======>......................] - ETA: 2s - loss: 0.0802 - accuracy: 0.9740
113/375 [========>.....................] - ETA: 2s - loss: 0.0809 - accuracy: 0.9738
119/375 [========>.....................] - ETA: 2s - loss: 0.0810 - accuracy: 0.9737
124/375 [========>.....................] - ETA: 2s - loss: 0.0805 - accuracy: 0.9740
130/375 [=========>....................] - ETA: 2s - loss: 0.0814 - accuracy: 0.9738
136/375 [=========>....................] - ETA: 2s - loss: 0.0812 - accuracy: 0.9739
142/375 [==========>...................] - ETA: 2s - loss: 0.0810 - accuracy: 0.9738
148/375 [==========>...................] - ETA: 2s - loss: 0.0815 - accuracy: 0.9739
155/375 [===========>..................] - ETA: 2s - loss: 0.0811 - accuracy: 0.9740
162/375 [===========>..................] - ETA: 2s - loss: 0.0817 - accuracy: 0.9737
169/375 [============>.................] - ETA: 1s - loss: 0.0820 - accuracy: 0.9736
176/375 [=============>................] - ETA: 1s - loss: 0.0828 - accuracy: 0.9735
183/375 [=============>................] - ETA: 1s - loss: 0.0838 - accuracy: 0.9734
190/375 [==============>...............] - ETA: 1s - loss: 0.0848 - accuracy: 0.9731
197/375 [==============>...............] - ETA: 1s - loss: 0.0844 - accuracy: 0.9732
203/375 [===============>..............] - ETA: 1s - loss: 0.0839 - accuracy: 0.9733
210/375 [===============>..............] - ETA: 1s - loss: 0.0837 - accuracy: 0.9735
216/375 [================>.............] - ETA: 1s - loss: 0.0833 - accuracy: 0.9735
223/375 [================>.............] - ETA: 1s - loss: 0.0834 - accuracy: 0.9733
229/375 [=================>............] - ETA: 1s - loss: 0.0833 - accuracy: 0.9733
236/375 [=================>............] - ETA: 1s - loss: 0.0832 - accuracy: 0.9733
243/375 [==================>...........] - ETA: 1s - loss: 0.0834 - accuracy: 0.9734
250/375 [===================>..........] - ETA: 1s - loss: 0.0827 - accuracy: 0.9735
257/375 [===================>..........] - ETA: 1s - loss: 0.0825 - accuracy: 0.9736
264/375 [====================>.........] - ETA: 0s - loss: 0.0825 - accuracy: 0.9736
271/375 [====================>.........] - ETA: 0s - loss: 0.0822 - accuracy: 0.9737
277/375 [=====================>........] - ETA: 0s - loss: 0.0819 - accuracy: 0.9738
284/375 [=====================>........] - ETA: 0s - loss: 0.0814 - accuracy: 0.9740
291/375 [======================>.......] - ETA: 0s - loss: 0.0819 - accuracy: 0.9739
298/375 [======================>.......] - ETA: 0s - loss: 0.0821 - accuracy: 0.9738
304/375 [=======================>......] - ETA: 0s - loss: 0.0820 - accuracy: 0.9739
311/375 [=======================>......] - ETA: 0s - loss: 0.0821 - accuracy: 0.9738
317/375 [========================>.....] - ETA: 0s - loss: 0.0818 - accuracy: 0.9739
321/375 [========================>.....] - ETA: 0s - loss: 0.0815 - accuracy: 0.9739
328/375 [=========================>....] - ETA: 0s - loss: 0.0817 - accuracy: 0.9738
335/375 [=========================>....] - ETA: 0s - loss: 0.0815 - accuracy: 0.9739
342/375 [==========================>...] - ETA: 0s - loss: 0.0811 - accuracy: 0.9741
349/375 [==========================>...] - ETA: 0s - loss: 0.0810 - accuracy: 0.9741
356/375 [===========================>..] - ETA: 0s - loss: 0.0813 - accuracy: 0.9740
362/375 [===========================>..] - ETA: 0s - loss: 0.0814 - accuracy: 0.9740
368/375 [============================>.] - ETA: 0s - loss: 0.0817 - accuracy: 0.9740
375/375 [==============================] - 3s 9ms/step - loss: 0.0819 - accuracy: 0.9738

375/375 [==============================] - 4s 12ms/step - loss: 0.0819 - accuracy: 0.9738 - val_loss: 0.1031 - val_accuracy: 0.9687 - lr: 0.1000
Epoch 4/20

  1/375 [..............................] - ETA: 2s - loss: 0.0567 - accuracy: 0.9922
  7/375 [..............................] - ETA: 3s - loss: 0.0539 - accuracy: 0.9844
 14/375 [>.............................] - ETA: 3s - loss: 0.0579 - accuracy: 0.9827
 20/375 [>.............................] - ETA: 2s - loss: 0.0624 - accuracy: 0.9809
 27/375 [=>............................] - ETA: 2s - loss: 0.0571 - accuracy: 0.9821
 34/375 [=>............................] - ETA: 2s - loss: 0.0597 - accuracy: 0.9812
 41/375 [==>...........................] - ETA: 2s - loss: 0.0598 - accuracy: 0.9808
 47/375 [==>...........................] - ETA: 2s - loss: 0.0614 - accuracy: 0.9802
 53/375 [===>..........................] - ETA: 2s - loss: 0.0601 - accuracy: 0.9813
 58/375 [===>..........................] - ETA: 2s - loss: 0.0592 - accuracy: 0.9813
 63/375 [====>.........................] - ETA: 2s - loss: 0.0590 - accuracy: 0.9812
 69/375 [====>.........................] - ETA: 2s - loss: 0.0588 - accuracy: 0.9812
 75/375 [=====>........................] - ETA: 2s - loss: 0.0574 - accuracy: 0.9818
 81/375 [=====>........................] - ETA: 2s - loss: 0.0576 - accuracy: 0.9818
 87/375 [=====>........................] - ETA: 2s - loss: 0.0589 - accuracy: 0.9811
 93/375 [======>.......................] - ETA: 2s - loss: 0.0580 - accuracy: 0.9814
 99/375 [======>.......................] - ETA: 2s - loss: 0.0568 - accuracy: 0.9818
105/375 [=======>......................] - ETA: 2s - loss: 0.0560 - accuracy: 0.9819
111/375 [=======>......................] - ETA: 2s - loss: 0.0566 - accuracy: 0.9818
117/375 [========>.....................] - ETA: 2s - loss: 0.0584 - accuracy: 0.9816
124/375 [========>.....................] - ETA: 2s - loss: 0.0582 - accuracy: 0.9817
129/375 [=========>....................] - ETA: 2s - loss: 0.0578 - accuracy: 0.9817
135/375 [=========>....................] - ETA: 2s - loss: 0.0580 - accuracy: 0.9815
141/375 [==========>...................] - ETA: 2s - loss: 0.0592 - accuracy: 0.9813
146/375 [==========>...................] - ETA: 2s - loss: 0.0595 - accuracy: 0.9811
152/375 [===========>..................] - ETA: 2s - loss: 0.0607 - accuracy: 0.9804
158/375 [===========>..................] - ETA: 1s - loss: 0.0604 - accuracy: 0.9803
164/375 [============>.................] - ETA: 1s - loss: 0.0605 - accuracy: 0.9801
170/375 [============>.................] - ETA: 1s - loss: 0.0607 - accuracy: 0.9799
176/375 [=============>................] - ETA: 1s - loss: 0.0606 - accuracy: 0.9801
182/375 [=============>................] - ETA: 1s - loss: 0.0611 - accuracy: 0.9800
188/375 [==============>...............] - ETA: 1s - loss: 0.0615 - accuracy: 0.9799
194/375 [==============>...............] - ETA: 1s - loss: 0.0623 - accuracy: 0.9798
199/375 [==============>...............] - ETA: 1s - loss: 0.0630 - accuracy: 0.9795
205/375 [===============>..............] - ETA: 1s - loss: 0.0629 - accuracy: 0.9796
211/375 [===============>..............] - ETA: 1s - loss: 0.0632 - accuracy: 0.9795
217/375 [================>.............] - ETA: 1s - loss: 0.0628 - accuracy: 0.9795
224/375 [================>.............] - ETA: 1s - loss: 0.0624 - accuracy: 0.9796
231/375 [=================>............] - ETA: 1s - loss: 0.0629 - accuracy: 0.9795
237/375 [=================>............] - ETA: 1s - loss: 0.0628 - accuracy: 0.9795
243/375 [==================>...........] - ETA: 1s - loss: 0.0635 - accuracy: 0.9794
249/375 [==================>...........] - ETA: 1s - loss: 0.0639 - accuracy: 0.9793
255/375 [===================>..........] - ETA: 1s - loss: 0.0638 - accuracy: 0.9794
261/375 [===================>..........] - ETA: 1s - loss: 0.0641 - accuracy: 0.9793
268/375 [====================>.........] - ETA: 0s - loss: 0.0639 - accuracy: 0.9794
274/375 [====================>.........] - ETA: 0s - loss: 0.0637 - accuracy: 0.9794
280/375 [=====================>........] - ETA: 0s - loss: 0.0643 - accuracy: 0.9791
287/375 [=====================>........] - ETA: 0s - loss: 0.0647 - accuracy: 0.9791
293/375 [======================>.......] - ETA: 0s - loss: 0.0647 - accuracy: 0.9791
300/375 [=======================>......] - ETA: 0s - loss: 0.0648 - accuracy: 0.9790
306/375 [=======================>......] - ETA: 0s - loss: 0.0653 - accuracy: 0.9788
312/375 [=======================>......] - ETA: 0s - loss: 0.0652 - accuracy: 0.9788
319/375 [========================>.....] - ETA: 0s - loss: 0.0652 - accuracy: 0.9789
325/375 [=========================>....] - ETA: 0s - loss: 0.0653 - accuracy: 0.9789
331/375 [=========================>....] - ETA: 0s - loss: 0.0653 - accuracy: 0.9789
337/375 [=========================>....] - ETA: 0s - loss: 0.0654 - accuracy: 0.9788
343/375 [==========================>...] - ETA: 0s - loss: 0.0654 - accuracy: 0.9788
349/375 [==========================>...] - ETA: 0s - loss: 0.0655 - accuracy: 0.9787
354/375 [===========================>..] - ETA: 0s - loss: 0.0652 - accuracy: 0.9788
360/375 [===========================>..] - ETA: 0s - loss: 0.0649 - accuracy: 0.9788
365/375 [============================>.] - ETA: 0s - loss: 0.0648 - accuracy: 0.9788
370/375 [============================>.] - ETA: 0s - loss: 0.0647 - accuracy: 0.9789
375/375 [==============================] - 3s 9ms/step - loss: 0.0647 - accuracy: 0.9789

375/375 [==============================] - 4s 12ms/step - loss: 0.0647 - accuracy: 0.9789 - val_loss: 0.1124 - val_accuracy: 0.9681 - lr: 0.1000
Epoch 5/20

  1/375 [..............................] - ETA: 3s - loss: 0.0423 - accuracy: 0.9844
  7/375 [..............................] - ETA: 3s - loss: 0.0602 - accuracy: 0.9788
 13/375 [>.............................] - ETA: 3s - loss: 0.0778 - accuracy: 0.9736
 20/375 [>.............................] - ETA: 2s - loss: 0.0743 - accuracy: 0.9773
 22/375 [>.............................] - ETA: 3s - loss: 0.0724 - accuracy: 0.9773
 27/375 [=>............................] - ETA: 3s - loss: 0.0725 - accuracy: 0.9771
 34/375 [=>............................] - ETA: 3s - loss: 0.0709 - accuracy: 0.9773
 39/375 [==>...........................] - ETA: 3s - loss: 0.0676 - accuracy: 0.9786
 45/375 [==>...........................] - ETA: 3s - loss: 0.0646 - accuracy: 0.9795
 52/375 [===>..........................] - ETA: 3s - loss: 0.0594 - accuracy: 0.9812
 58/375 [===>..........................] - ETA: 3s - loss: 0.0563 - accuracy: 0.9822
 64/375 [====>.........................] - ETA: 3s - loss: 0.0548 - accuracy: 0.9828
 71/375 [====>.........................] - ETA: 2s - loss: 0.0522 - accuracy: 0.9836
 78/375 [=====>........................] - ETA: 2s - loss: 0.0502 - accuracy: 0.9844
 83/375 [=====>........................] - ETA: 2s - loss: 0.0508 - accuracy: 0.9838
 90/375 [======>.......................] - ETA: 2s - loss: 0.0489 - accuracy: 0.9845
 96/375 [======>.......................] - ETA: 2s - loss: 0.0474 - accuracy: 0.9850
102/375 [=======>......................] - ETA: 2s - loss: 0.0467 - accuracy: 0.9849
109/375 [=======>......................] - ETA: 2s - loss: 0.0461 - accuracy: 0.9852
115/375 [========>.....................] - ETA: 2s - loss: 0.0463 - accuracy: 0.9855
121/375 [========>.....................] - ETA: 2s - loss: 0.0456 - accuracy: 0.9856
127/375 [=========>....................] - ETA: 2s - loss: 0.0448 - accuracy: 0.9860
133/375 [=========>....................] - ETA: 2s - loss: 0.0436 - accuracy: 0.9865
140/375 [==========>...................] - ETA: 2s - loss: 0.0426 - accuracy: 0.9869
147/375 [==========>...................] - ETA: 2s - loss: 0.0419 - accuracy: 0.9872
153/375 [===========>..................] - ETA: 2s - loss: 0.0417 - accuracy: 0.9873
159/375 [===========>..................] - ETA: 1s - loss: 0.0406 - accuracy: 0.9878
166/375 [============>.................] - ETA: 1s - loss: 0.0399 - accuracy: 0.9881
172/375 [============>.................] - ETA: 1s - loss: 0.0399 - accuracy: 0.9881
179/375 [=============>................] - ETA: 1s - loss: 0.0397 - accuracy: 0.9880
185/375 [=============>................] - ETA: 1s - loss: 0.0396 - accuracy: 0.9879
191/375 [==============>...............] - ETA: 1s - loss: 0.0390 - accuracy: 0.9881
198/375 [==============>...............] - ETA: 1s - loss: 0.0389 - accuracy: 0.9880
204/375 [===============>..............] - ETA: 1s - loss: 0.0384 - accuracy: 0.9881
210/375 [===============>..............] - ETA: 1s - loss: 0.0383 - accuracy: 0.9881
216/375 [================>.............] - ETA: 1s - loss: 0.0377 - accuracy: 0.9883
222/375 [================>.............] - ETA: 1s - loss: 0.0373 - accuracy: 0.9884
228/375 [=================>............] - ETA: 1s - loss: 0.0368 - accuracy: 0.9886
234/375 [=================>............] - ETA: 1s - loss: 0.0364 - accuracy: 0.9886
240/375 [==================>...........] - ETA: 1s - loss: 0.0360 - accuracy: 0.9888
246/375 [==================>...........] - ETA: 1s - loss: 0.0362 - accuracy: 0.9887
253/375 [===================>..........] - ETA: 1s - loss: 0.0361 - accuracy: 0.9887
259/375 [===================>..........] - ETA: 1s - loss: 0.0359 - accuracy: 0.9887
265/375 [====================>.........] - ETA: 0s - loss: 0.0356 - accuracy: 0.9888
272/375 [====================>.........] - ETA: 0s - loss: 0.0354 - accuracy: 0.9889
279/375 [=====================>........] - ETA: 0s - loss: 0.0352 - accuracy: 0.9890
286/375 [=====================>........] - ETA: 0s - loss: 0.0352 - accuracy: 0.9890
293/375 [======================>.......] - ETA: 0s - loss: 0.0350 - accuracy: 0.9890
300/375 [=======================>......] - ETA: 0s - loss: 0.0348 - accuracy: 0.9891
307/375 [=======================>......] - ETA: 0s - loss: 0.0348 - accuracy: 0.9891
314/375 [========================>.....] - ETA: 0s - loss: 0.0352 - accuracy: 0.9891
321/375 [========================>.....] - ETA: 0s - loss: 0.0350 - accuracy: 0.9892
328/375 [=========================>....] - ETA: 0s - loss: 0.0347 - accuracy: 0.9892
334/375 [=========================>....] - ETA: 0s - loss: 0.0345 - accuracy: 0.9894
341/375 [==========================>...] - ETA: 0s - loss: 0.0343 - accuracy: 0.9894
347/375 [==========================>...] - ETA: 0s - loss: 0.0340 - accuracy: 0.9895
353/375 [===========================>..] - ETA: 0s - loss: 0.0337 - accuracy: 0.9896
359/375 [===========================>..] - ETA: 0s - loss: 0.0337 - accuracy: 0.9896
366/375 [============================>.] - ETA: 0s - loss: 0.0336 - accuracy: 0.9896
371/375 [============================>.] - ETA: 0s - loss: 0.0336 - accuracy: 0.9896
373/375 [============================>.] - ETA: 0s - loss: 0.0336 - accuracy: 0.9896
375/375 [==============================] - 3s 9ms/step - loss: 0.0335 - accuracy: 0.9896

375/375 [==============================] - 4s 11ms/step - loss: 0.0335 - accuracy: 0.9896 - val_loss: 0.0815 - val_accuracy: 0.9762 - lr: 0.0100
Epoch 6/20

  1/375 [..............................] - ETA: 3s - loss: 0.0217 - accuracy: 0.9922
  8/375 [..............................] - ETA: 3s - loss: 0.0205 - accuracy: 0.9951
 15/375 [>.............................] - ETA: 3s - loss: 0.0248 - accuracy: 0.9943
 21/375 [>.............................] - ETA: 2s - loss: 0.0244 - accuracy: 0.9944
 28/375 [=>............................] - ETA: 2s - loss: 0.0229 - accuracy: 0.9941
 34/375 [=>............................] - ETA: 2s - loss: 0.0269 - accuracy: 0.9926
 40/375 [==>...........................] - ETA: 2s - loss: 0.0251 - accuracy: 0.9936
 46/375 [==>...........................] - ETA: 2s - loss: 0.0241 - accuracy: 0.9939
 51/375 [===>..........................] - ETA: 2s - loss: 0.0241 - accuracy: 0.9940
 56/375 [===>..........................] - ETA: 2s - loss: 0.0239 - accuracy: 0.9940
 62/375 [===>..........................] - ETA: 2s - loss: 0.0228 - accuracy: 0.9945
 68/375 [====>.........................] - ETA: 2s - loss: 0.0222 - accuracy: 0.9947
 74/375 [====>.........................] - ETA: 2s - loss: 0.0221 - accuracy: 0.9944
 81/375 [=====>........................] - ETA: 2s - loss: 0.0231 - accuracy: 0.9939
 87/375 [=====>........................] - ETA: 2s - loss: 0.0231 - accuracy: 0.9939
 93/375 [======>.......................] - ETA: 2s - loss: 0.0231 - accuracy: 0.9940
 99/375 [======>.......................] - ETA: 2s - loss: 0.0227 - accuracy: 0.9940
106/375 [=======>......................] - ETA: 2s - loss: 0.0218 - accuracy: 0.9943
112/375 [=======>......................] - ETA: 2s - loss: 0.0223 - accuracy: 0.9941
118/375 [========>.....................] - ETA: 2s - loss: 0.0220 - accuracy: 0.9943
124/375 [========>.....................] - ETA: 2s - loss: 0.0220 - accuracy: 0.9942
130/375 [=========>....................] - ETA: 2s - loss: 0.0220 - accuracy: 0.9942
136/375 [=========>....................] - ETA: 2s - loss: 0.0224 - accuracy: 0.9939
142/375 [==========>...................] - ETA: 2s - loss: 0.0224 - accuracy: 0.9941
148/375 [==========>...................] - ETA: 2s - loss: 0.0221 - accuracy: 0.9942
154/375 [===========>..................] - ETA: 1s - loss: 0.0221 - accuracy: 0.9943
161/375 [===========>..................] - ETA: 1s - loss: 0.0223 - accuracy: 0.9942
167/375 [============>.................] - ETA: 1s - loss: 0.0224 - accuracy: 0.9941
173/375 [============>.................] - ETA: 1s - loss: 0.0222 - accuracy: 0.9940
179/375 [=============>................] - ETA: 1s - loss: 0.0220 - accuracy: 0.9941
186/375 [=============>................] - ETA: 1s - loss: 0.0216 - accuracy: 0.9942
192/375 [==============>...............] - ETA: 1s - loss: 0.0215 - accuracy: 0.9942
199/375 [==============>...............] - ETA: 1s - loss: 0.0218 - accuracy: 0.9942
206/375 [===============>..............] - ETA: 1s - loss: 0.0220 - accuracy: 0.9940
212/375 [===============>..............] - ETA: 1s - loss: 0.0218 - accuracy: 0.9940
218/375 [================>.............] - ETA: 1s - loss: 0.0218 - accuracy: 0.9940
225/375 [=================>............] - ETA: 1s - loss: 0.0218 - accuracy: 0.9940
232/375 [=================>............] - ETA: 1s - loss: 0.0218 - accuracy: 0.9940
238/375 [==================>...........] - ETA: 1s - loss: 0.0217 - accuracy: 0.9940
245/375 [==================>...........] - ETA: 1s - loss: 0.0215 - accuracy: 0.9941
251/375 [===================>..........] - ETA: 1s - loss: 0.0216 - accuracy: 0.9941
257/375 [===================>..........] - ETA: 1s - loss: 0.0214 - accuracy: 0.9942
263/375 [====================>.........] - ETA: 0s - loss: 0.0219 - accuracy: 0.9941
269/375 [====================>.........] - ETA: 0s - loss: 0.0216 - accuracy: 0.9942
276/375 [=====================>........] - ETA: 0s - loss: 0.0217 - accuracy: 0.9941
282/375 [=====================>........] - ETA: 0s - loss: 0.0215 - accuracy: 0.9941
288/375 [======================>.......] - ETA: 0s - loss: 0.0216 - accuracy: 0.9941
294/375 [======================>.......] - ETA: 0s - loss: 0.0216 - accuracy: 0.9940
301/375 [=======================>......] - ETA: 0s - loss: 0.0217 - accuracy: 0.9939
307/375 [=======================>......] - ETA: 0s - loss: 0.0218 - accuracy: 0.9939
313/375 [========================>.....] - ETA: 0s - loss: 0.0217 - accuracy: 0.9939
320/375 [========================>.....] - ETA: 0s - loss: 0.0219 - accuracy: 0.9938
326/375 [=========================>....] - ETA: 0s - loss: 0.0222 - accuracy: 0.9937
333/375 [=========================>....] - ETA: 0s - loss: 0.0222 - accuracy: 0.9937
339/375 [==========================>...] - ETA: 0s - loss: 0.0220 - accuracy: 0.9937
346/375 [==========================>...] - ETA: 0s - loss: 0.0220 - accuracy: 0.9937
353/375 [===========================>..] - ETA: 0s - loss: 0.0221 - accuracy: 0.9937
359/375 [===========================>..] - ETA: 0s - loss: 0.0223 - accuracy: 0.9937
366/375 [============================>.] - ETA: 0s - loss: 0.0223 - accuracy: 0.9937
373/375 [============================>.] - ETA: 0s - loss: 0.0222 - accuracy: 0.9938
375/375 [==============================] - 3s 9ms/step - loss: 0.0221 - accuracy: 0.9938

375/375 [==============================] - 4s 11ms/step - loss: 0.0221 - accuracy: 0.9938 - val_loss: 0.0772 - val_accuracy: 0.9781 - lr: 0.0100
Epoch 7/20

  1/375 [..............................] - ETA: 2s - loss: 0.0069 - accuracy: 1.0000
  7/375 [..............................] - ETA: 3s - loss: 0.0130 - accuracy: 0.9978
 13/375 [>.............................] - ETA: 3s - loss: 0.0160 - accuracy: 0.9964
 20/375 [>.............................] - ETA: 3s - loss: 0.0134 - accuracy: 0.9977
 27/375 [=>............................] - ETA: 2s - loss: 0.0145 - accuracy: 0.9977
 33/375 [=>............................] - ETA: 2s - loss: 0.0183 - accuracy: 0.9964
 39/375 [==>...........................] - ETA: 2s - loss: 0.0196 - accuracy: 0.9960
 46/375 [==>...........................] - ETA: 2s - loss: 0.0189 - accuracy: 0.9956
 53/375 [===>..........................] - ETA: 2s - loss: 0.0177 - accuracy: 0.9960
 59/375 [===>..........................] - ETA: 2s - loss: 0.0184 - accuracy: 0.9960
 65/375 [====>.........................] - ETA: 2s - loss: 0.0177 - accuracy: 0.9964
 71/375 [====>.........................] - ETA: 2s - loss: 0.0177 - accuracy: 0.9964
 77/375 [=====>........................] - ETA: 2s - loss: 0.0182 - accuracy: 0.9963
 84/375 [=====>........................] - ETA: 2s - loss: 0.0182 - accuracy: 0.9965
 90/375 [======>.......................] - ETA: 2s - loss: 0.0187 - accuracy: 0.9960
 97/375 [======>.......................] - ETA: 2s - loss: 0.0186 - accuracy: 0.9961
104/375 [=======>......................] - ETA: 2s - loss: 0.0185 - accuracy: 0.9961
110/375 [=======>......................] - ETA: 2s - loss: 0.0185 - accuracy: 0.9961
117/375 [========>.....................] - ETA: 2s - loss: 0.0186 - accuracy: 0.9960
123/375 [========>.....................] - ETA: 2s - loss: 0.0189 - accuracy: 0.9957
129/375 [=========>....................] - ETA: 2s - loss: 0.0188 - accuracy: 0.9956
136/375 [=========>....................] - ETA: 2s - loss: 0.0185 - accuracy: 0.9956
143/375 [==========>...................] - ETA: 1s - loss: 0.0189 - accuracy: 0.9954
150/375 [===========>..................] - ETA: 1s - loss: 0.0191 - accuracy: 0.9953
156/375 [===========>..................] - ETA: 1s - loss: 0.0190 - accuracy: 0.9953
163/375 [============>.................] - ETA: 1s - loss: 0.0192 - accuracy: 0.9953
170/375 [============>.................] - ETA: 1s - loss: 0.0189 - accuracy: 0.9954
176/375 [=============>................] - ETA: 1s - loss: 0.0188 - accuracy: 0.9954
183/375 [=============>................] - ETA: 1s - loss: 0.0187 - accuracy: 0.9953
189/375 [==============>...............] - ETA: 1s - loss: 0.0189 - accuracy: 0.9952
196/375 [==============>...............] - ETA: 1s - loss: 0.0191 - accuracy: 0.9952
203/375 [===============>..............] - ETA: 1s - loss: 0.0188 - accuracy: 0.9953
209/375 [===============>..............] - ETA: 1s - loss: 0.0188 - accuracy: 0.9954
216/375 [================>.............] - ETA: 1s - loss: 0.0189 - accuracy: 0.9953
223/375 [================>.............] - ETA: 1s - loss: 0.0188 - accuracy: 0.9953
229/375 [=================>............] - ETA: 1s - loss: 0.0187 - accuracy: 0.9953
235/375 [=================>............] - ETA: 1s - loss: 0.0184 - accuracy: 0.9954
242/375 [==================>...........] - ETA: 1s - loss: 0.0182 - accuracy: 0.9954
247/375 [==================>...........] - ETA: 1s - loss: 0.0183 - accuracy: 0.9954
253/375 [===================>..........] - ETA: 1s - loss: 0.0182 - accuracy: 0.9954
259/375 [===================>..........] - ETA: 0s - loss: 0.0181 - accuracy: 0.9954
265/375 [====================>.........] - ETA: 0s - loss: 0.0183 - accuracy: 0.9953
271/375 [====================>.........] - ETA: 0s - loss: 0.0184 - accuracy: 0.9953
276/375 [=====================>........] - ETA: 0s - loss: 0.0187 - accuracy: 0.9951
282/375 [=====================>........] - ETA: 0s - loss: 0.0190 - accuracy: 0.9950
288/375 [======================>.......] - ETA: 0s - loss: 0.0190 - accuracy: 0.9950
294/375 [======================>.......] - ETA: 0s - loss: 0.0190 - accuracy: 0.9950
300/375 [=======================>......] - ETA: 0s - loss: 0.0192 - accuracy: 0.9950
307/375 [=======================>......] - ETA: 0s - loss: 0.0193 - accuracy: 0.9949
313/375 [========================>.....] - ETA: 0s - loss: 0.0193 - accuracy: 0.9949
320/375 [========================>.....] - ETA: 0s - loss: 0.0191 - accuracy: 0.9949
326/375 [=========================>....] - ETA: 0s - loss: 0.0191 - accuracy: 0.9949
332/375 [=========================>....] - ETA: 0s - loss: 0.0191 - accuracy: 0.9950
338/375 [==========================>...] - ETA: 0s - loss: 0.0191 - accuracy: 0.9950
344/375 [==========================>...] - ETA: 0s - loss: 0.0191 - accuracy: 0.9949
350/375 [===========================>..] - ETA: 0s - loss: 0.0193 - accuracy: 0.9948
357/375 [===========================>..] - ETA: 0s - loss: 0.0192 - accuracy: 0.9949
362/375 [===========================>..] - ETA: 0s - loss: 0.0192 - accuracy: 0.9949
369/375 [============================>.] - ETA: 0s - loss: 0.0192 - accuracy: 0.9949
375/375 [==============================] - 3s 9ms/step - loss: 0.0190 - accuracy: 0.9950

375/375 [==============================] - 4s 11ms/step - loss: 0.0190 - accuracy: 0.9950 - val_loss: 0.0773 - val_accuracy: 0.9785 - lr: 0.0100
Epoch 8/20

  1/375 [..............................] - ETA: 2s - loss: 0.0146 - accuracy: 1.0000
  8/375 [..............................] - ETA: 2s - loss: 0.0098 - accuracy: 0.9980
 15/375 [>.............................] - ETA: 2s - loss: 0.0122 - accuracy: 0.9969
 21/375 [>.............................] - ETA: 2s - loss: 0.0118 - accuracy: 0.9967
 27/375 [=>............................] - ETA: 2s - loss: 0.0154 - accuracy: 0.9962
 32/375 [=>............................] - ETA: 3s - loss: 0.0152 - accuracy: 0.9961
 38/375 [==>...........................] - ETA: 2s - loss: 0.0168 - accuracy: 0.9953
 44/375 [==>...........................] - ETA: 2s - loss: 0.0159 - accuracy: 0.9959
 50/375 [===>..........................] - ETA: 2s - loss: 0.0154 - accuracy: 0.9962
 56/375 [===>..........................] - ETA: 2s - loss: 0.0153 - accuracy: 0.9961
 61/375 [===>..........................] - ETA: 2s - loss: 0.0153 - accuracy: 0.9962
 67/375 [====>.........................] - ETA: 2s - loss: 0.0152 - accuracy: 0.9962
 73/375 [====>.........................] - ETA: 2s - loss: 0.0152 - accuracy: 0.9964
 80/375 [=====>........................] - ETA: 2s - loss: 0.0162 - accuracy: 0.9964
 87/375 [=====>........................] - ETA: 2s - loss: 0.0164 - accuracy: 0.9966
 93/375 [======>.......................] - ETA: 2s - loss: 0.0162 - accuracy: 0.9966
 99/375 [======>.......................] - ETA: 2s - loss: 0.0165 - accuracy: 0.9964
105/375 [=======>......................] - ETA: 2s - loss: 0.0166 - accuracy: 0.9963
111/375 [=======>......................] - ETA: 2s - loss: 0.0167 - accuracy: 0.9963
117/375 [========>.....................] - ETA: 2s - loss: 0.0172 - accuracy: 0.9961
123/375 [========>.....................] - ETA: 2s - loss: 0.0171 - accuracy: 0.9963
127/375 [=========>....................] - ETA: 2s - loss: 0.0170 - accuracy: 0.9962
134/375 [=========>....................] - ETA: 2s - loss: 0.0168 - accuracy: 0.9962
140/375 [==========>...................] - ETA: 2s - loss: 0.0172 - accuracy: 0.9961
146/375 [==========>...................] - ETA: 2s - loss: 0.0170 - accuracy: 0.9962
152/375 [===========>..................] - ETA: 1s - loss: 0.0167 - accuracy: 0.9963
159/375 [===========>..................] - ETA: 1s - loss: 0.0166 - accuracy: 0.9963
165/375 [============>.................] - ETA: 1s - loss: 0.0171 - accuracy: 0.9962
172/375 [============>.................] - ETA: 1s - loss: 0.0171 - accuracy: 0.9962
178/375 [=============>................] - ETA: 1s - loss: 0.0172 - accuracy: 0.9962
184/375 [=============>................] - ETA: 1s - loss: 0.0174 - accuracy: 0.9961
190/375 [==============>...............] - ETA: 1s - loss: 0.0177 - accuracy: 0.9961
196/375 [==============>...............] - ETA: 1s - loss: 0.0177 - accuracy: 0.9961
203/375 [===============>..............] - ETA: 1s - loss: 0.0176 - accuracy: 0.9961
209/375 [===============>..............] - ETA: 1s - loss: 0.0178 - accuracy: 0.9960
215/375 [================>.............] - ETA: 1s - loss: 0.0176 - accuracy: 0.9961
222/375 [================>.............] - ETA: 1s - loss: 0.0179 - accuracy: 0.9960
229/375 [=================>............] - ETA: 1s - loss: 0.0177 - accuracy: 0.9961
235/375 [=================>............] - ETA: 1s - loss: 0.0177 - accuracy: 0.9961
242/375 [==================>...........] - ETA: 1s - loss: 0.0175 - accuracy: 0.9962
248/375 [==================>...........] - ETA: 1s - loss: 0.0174 - accuracy: 0.9962
254/375 [===================>..........] - ETA: 1s - loss: 0.0173 - accuracy: 0.9962
261/375 [===================>..........] - ETA: 0s - loss: 0.0171 - accuracy: 0.9962
267/375 [====================>.........] - ETA: 0s - loss: 0.0170 - accuracy: 0.9963
273/375 [====================>.........] - ETA: 0s - loss: 0.0171 - accuracy: 0.9962
280/375 [=====================>........] - ETA: 0s - loss: 0.0171 - accuracy: 0.9962
286/375 [=====================>........] - ETA: 0s - loss: 0.0171 - accuracy: 0.9962
293/375 [======================>.......] - ETA: 0s - loss: 0.0168 - accuracy: 0.9963
300/375 [=======================>......] - ETA: 0s - loss: 0.0167 - accuracy: 0.9963
307/375 [=======================>......] - ETA: 0s - loss: 0.0166 - accuracy: 0.9963
313/375 [========================>.....] - ETA: 0s - loss: 0.0166 - accuracy: 0.9963
319/375 [========================>.....] - ETA: 0s - loss: 0.0165 - accuracy: 0.9962
325/375 [=========================>....] - ETA: 0s - loss: 0.0167 - accuracy: 0.9961
331/375 [=========================>....] - ETA: 0s - loss: 0.0166 - accuracy: 0.9961
338/375 [==========================>...] - ETA: 0s - loss: 0.0165 - accuracy: 0.9962
344/375 [==========================>...] - ETA: 0s - loss: 0.0165 - accuracy: 0.9962
350/375 [===========================>..] - ETA: 0s - loss: 0.0164 - accuracy: 0.9962
356/375 [===========================>..] - ETA: 0s - loss: 0.0164 - accuracy: 0.9961
362/375 [===========================>..] - ETA: 0s - loss: 0.0164 - accuracy: 0.9961
368/375 [============================>.] - ETA: 0s - loss: 0.0164 - accuracy: 0.9961
375/375 [==============================] - 3s 9ms/step - loss: 0.0164 - accuracy: 0.9961

375/375 [==============================] - 4s 11ms/step - loss: 0.0164 - accuracy: 0.9961 - val_loss: 0.0761 - val_accuracy: 0.9786 - lr: 1.0000e-03
Epoch 9/20

  1/375 [..............................] - ETA: 2s - loss: 0.0086 - accuracy: 1.0000
  8/375 [..............................] - ETA: 2s - loss: 0.0195 - accuracy: 0.9951
 14/375 [>.............................] - ETA: 2s - loss: 0.0167 - accuracy: 0.9955
 21/375 [>.............................] - ETA: 2s - loss: 0.0152 - accuracy: 0.9955
 28/375 [=>............................] - ETA: 2s - loss: 0.0139 - accuracy: 0.9961
 35/375 [=>............................] - ETA: 2s - loss: 0.0127 - accuracy: 0.9969
 41/375 [==>...........................] - ETA: 2s - loss: 0.0137 - accuracy: 0.9964
 47/375 [==>...........................] - ETA: 2s - loss: 0.0134 - accuracy: 0.9963
 53/375 [===>..........................] - ETA: 2s - loss: 0.0132 - accuracy: 0.9965
 59/375 [===>..........................] - ETA: 2s - loss: 0.0150 - accuracy: 0.9962
 65/375 [====>.........................] - ETA: 2s - loss: 0.0153 - accuracy: 0.9960
 71/375 [====>.........................] - ETA: 2s - loss: 0.0147 - accuracy: 0.9963
 78/375 [=====>........................] - ETA: 2s - loss: 0.0146 - accuracy: 0.9964
 84/375 [=====>........................] - ETA: 2s - loss: 0.0149 - accuracy: 0.9964
 91/375 [======>.......................] - ETA: 2s - loss: 0.0146 - accuracy: 0.9965
 97/375 [======>.......................] - ETA: 2s - loss: 0.0140 - accuracy: 0.9967
104/375 [=======>......................] - ETA: 2s - loss: 0.0139 - accuracy: 0.9967
110/375 [=======>......................] - ETA: 2s - loss: 0.0145 - accuracy: 0.9964
116/375 [========>.....................] - ETA: 2s - loss: 0.0142 - accuracy: 0.9965
122/375 [========>.....................] - ETA: 2s - loss: 0.0144 - accuracy: 0.9963
128/375 [=========>....................] - ETA: 2s - loss: 0.0141 - accuracy: 0.9965
134/375 [=========>....................] - ETA: 2s - loss: 0.0150 - accuracy: 0.9963
140/375 [==========>...................] - ETA: 2s - loss: 0.0146 - accuracy: 0.9964
146/375 [==========>...................] - ETA: 1s - loss: 0.0143 - accuracy: 0.9966
152/375 [===========>..................] - ETA: 1s - loss: 0.0145 - accuracy: 0.9965
158/375 [===========>..................] - ETA: 1s - loss: 0.0148 - accuracy: 0.9964
165/375 [============>.................] - ETA: 1s - loss: 0.0149 - accuracy: 0.9964
171/375 [============>.................] - ETA: 1s - loss: 0.0150 - accuracy: 0.9963
178/375 [=============>................] - ETA: 1s - loss: 0.0151 - accuracy: 0.9963
185/375 [=============>................] - ETA: 1s - loss: 0.0153 - accuracy: 0.9963
191/375 [==============>...............] - ETA: 1s - loss: 0.0153 - accuracy: 0.9964
197/375 [==============>...............] - ETA: 1s - loss: 0.0153 - accuracy: 0.9964
204/375 [===============>..............] - ETA: 1s - loss: 0.0155 - accuracy: 0.9963
209/375 [===============>..............] - ETA: 1s - loss: 0.0154 - accuracy: 0.9963
215/375 [================>.............] - ETA: 1s - loss: 0.0156 - accuracy: 0.9963
222/375 [================>.............] - ETA: 1s - loss: 0.0154 - accuracy: 0.9963
228/375 [=================>............] - ETA: 1s - loss: 0.0152 - accuracy: 0.9964
235/375 [=================>............] - ETA: 1s - loss: 0.0151 - accuracy: 0.9964
241/375 [==================>...........] - ETA: 1s - loss: 0.0151 - accuracy: 0.9964
248/375 [==================>...........] - ETA: 1s - loss: 0.0152 - accuracy: 0.9963
254/375 [===================>..........] - ETA: 1s - loss: 0.0158 - accuracy: 0.9963
261/375 [===================>..........] - ETA: 0s - loss: 0.0158 - accuracy: 0.9963
267/375 [====================>.........] - ETA: 0s - loss: 0.0158 - accuracy: 0.9963
269/375 [====================>.........] - ETA: 0s - loss: 0.0158 - accuracy: 0.9964
274/375 [====================>.........] - ETA: 0s - loss: 0.0159 - accuracy: 0.9963
280/375 [=====================>........] - ETA: 0s - loss: 0.0159 - accuracy: 0.9963
285/375 [=====================>........] - ETA: 0s - loss: 0.0158 - accuracy: 0.9964
292/375 [======================>.......] - ETA: 0s - loss: 0.0158 - accuracy: 0.9964
298/375 [======================>.......] - ETA: 0s - loss: 0.0159 - accuracy: 0.9964
304/375 [=======================>......] - ETA: 0s - loss: 0.0161 - accuracy: 0.9963
311/375 [=======================>......] - ETA: 0s - loss: 0.0161 - accuracy: 0.9963
318/375 [========================>.....] - ETA: 0s - loss: 0.0160 - accuracy: 0.9963
324/375 [========================>.....] - ETA: 0s - loss: 0.0159 - accuracy: 0.9963
331/375 [=========================>....] - ETA: 0s - loss: 0.0161 - accuracy: 0.9962
336/375 [=========================>....] - ETA: 0s - loss: 0.0161 - accuracy: 0.9962
342/375 [==========================>...] - ETA: 0s - loss: 0.0163 - accuracy: 0.9962
349/375 [==========================>...] - ETA: 0s - loss: 0.0163 - accuracy: 0.9962
355/375 [===========================>..] - ETA: 0s - loss: 0.0162 - accuracy: 0.9962
361/375 [===========================>..] - ETA: 0s - loss: 0.0161 - accuracy: 0.9963
367/375 [============================>.] - ETA: 0s - loss: 0.0160 - accuracy: 0.9963
373/375 [============================>.] - ETA: 0s - loss: 0.0161 - accuracy: 0.9963
375/375 [==============================] - 3s 9ms/step - loss: 0.0160 - accuracy: 0.9963

375/375 [==============================] - 4s 11ms/step - loss: 0.0160 - accuracy: 0.9963 - val_loss: 0.0759 - val_accuracy: 0.9787 - lr: 1.0000e-03
Epoch 10/20

  1/375 [..............................] - ETA: 3s - loss: 0.0108 - accuracy: 1.0000
  7/375 [..............................] - ETA: 3s - loss: 0.0107 - accuracy: 0.9967
 12/375 [..............................] - ETA: 3s - loss: 0.0118 - accuracy: 0.9961
 18/375 [>.............................] - ETA: 3s - loss: 0.0105 - accuracy: 0.9970
 25/375 [=>............................] - ETA: 3s - loss: 0.0154 - accuracy: 0.9959
 32/375 [=>............................] - ETA: 3s - loss: 0.0166 - accuracy: 0.9954
 38/375 [==>...........................] - ETA: 3s - loss: 0.0165 - accuracy: 0.9955
 43/375 [==>...........................] - ETA: 3s - loss: 0.0164 - accuracy: 0.9956
 49/375 [==>...........................] - ETA: 3s - loss: 0.0177 - accuracy: 0.9954
 56/375 [===>..........................] - ETA: 2s - loss: 0.0176 - accuracy: 0.9950
 62/375 [===>..........................] - ETA: 2s - loss: 0.0171 - accuracy: 0.9953
 69/375 [====>.........................] - ETA: 2s - loss: 0.0175 - accuracy: 0.9954
 75/375 [=====>........................] - ETA: 2s - loss: 0.0179 - accuracy: 0.9951
 82/375 [=====>........................] - ETA: 2s - loss: 0.0172 - accuracy: 0.9953
 88/375 [======>.......................] - ETA: 2s - loss: 0.0169 - accuracy: 0.9955
 93/375 [======>.......................] - ETA: 2s - loss: 0.0165 - accuracy: 0.9955
 98/375 [======>.......................] - ETA: 2s - loss: 0.0166 - accuracy: 0.9955
104/375 [=======>......................] - ETA: 2s - loss: 0.0163 - accuracy: 0.9956
110/375 [=======>......................] - ETA: 2s - loss: 0.0164 - accuracy: 0.9957
116/375 [========>.....................] - ETA: 2s - loss: 0.0161 - accuracy: 0.9958
122/375 [========>.....................] - ETA: 2s - loss: 0.0163 - accuracy: 0.9957
128/375 [=========>....................] - ETA: 2s - loss: 0.0160 - accuracy: 0.9958
134/375 [=========>....................] - ETA: 2s - loss: 0.0157 - accuracy: 0.9960
141/375 [==========>...................] - ETA: 2s - loss: 0.0158 - accuracy: 0.9960
147/375 [==========>...................] - ETA: 2s - loss: 0.0157 - accuracy: 0.9961
153/375 [===========>..................] - ETA: 1s - loss: 0.0156 - accuracy: 0.9962
160/375 [===========>..................] - ETA: 1s - loss: 0.0157 - accuracy: 0.9963
166/375 [============>.................] - ETA: 1s - loss: 0.0157 - accuracy: 0.9964
173/375 [============>.................] - ETA: 1s - loss: 0.0154 - accuracy: 0.9964
179/375 [=============>................] - ETA: 1s - loss: 0.0156 - accuracy: 0.9964
185/375 [=============>................] - ETA: 1s - loss: 0.0159 - accuracy: 0.9962
192/375 [==============>...............] - ETA: 1s - loss: 0.0159 - accuracy: 0.9962
198/375 [==============>...............] - ETA: 1s - loss: 0.0157 - accuracy: 0.9963
204/375 [===============>..............] - ETA: 1s - loss: 0.0156 - accuracy: 0.9963
211/375 [===============>..............] - ETA: 1s - loss: 0.0156 - accuracy: 0.9963
218/375 [================>.............] - ETA: 1s - loss: 0.0157 - accuracy: 0.9963
225/375 [=================>............] - ETA: 1s - loss: 0.0161 - accuracy: 0.9963
232/375 [=================>............] - ETA: 1s - loss: 0.0159 - accuracy: 0.9964
239/375 [==================>...........] - ETA: 1s - loss: 0.0161 - accuracy: 0.9963
246/375 [==================>...........] - ETA: 1s - loss: 0.0162 - accuracy: 0.9963
253/375 [===================>..........] - ETA: 1s - loss: 0.0163 - accuracy: 0.9962
260/375 [===================>..........] - ETA: 0s - loss: 0.0161 - accuracy: 0.9963
267/375 [====================>.........] - ETA: 0s - loss: 0.0161 - accuracy: 0.9963
274/375 [====================>.........] - ETA: 0s - loss: 0.0160 - accuracy: 0.9964
281/375 [=====================>........] - ETA: 0s - loss: 0.0159 - accuracy: 0.9964
287/375 [=====================>........] - ETA: 0s - loss: 0.0160 - accuracy: 0.9963
294/375 [======================>.......] - ETA: 0s - loss: 0.0159 - accuracy: 0.9963
301/375 [=======================>......] - ETA: 0s - loss: 0.0160 - accuracy: 0.9963
308/375 [=======================>......] - ETA: 0s - loss: 0.0159 - accuracy: 0.9963
314/375 [========================>.....] - ETA: 0s - loss: 0.0158 - accuracy: 0.9963
320/375 [========================>.....] - ETA: 0s - loss: 0.0157 - accuracy: 0.9964
327/375 [=========================>....] - ETA: 0s - loss: 0.0158 - accuracy: 0.9963
332/375 [=========================>....] - ETA: 0s - loss: 0.0158 - accuracy: 0.9963
338/375 [==========================>...] - ETA: 0s - loss: 0.0156 - accuracy: 0.9963
344/375 [==========================>...] - ETA: 0s - loss: 0.0156 - accuracy: 0.9964
350/375 [===========================>..] - ETA: 0s - loss: 0.0155 - accuracy: 0.9964
356/375 [===========================>..] - ETA: 0s - loss: 0.0156 - accuracy: 0.9964
363/375 [============================>.] - ETA: 0s - loss: 0.0155 - accuracy: 0.9964
369/375 [============================>.] - ETA: 0s - loss: 0.0159 - accuracy: 0.9963
375/375 [==============================] - 3s 9ms/step - loss: 0.0158 - accuracy: 0.9963

375/375 [==============================] - 4s 11ms/step - loss: 0.0158 - accuracy: 0.9963 - val_loss: 0.0757 - val_accuracy: 0.9786 - lr: 1.0000e-03
Epoch 11/20

  1/375 [..............................] - ETA: 2s - loss: 0.0034 - accuracy: 1.0000
  8/375 [..............................] - ETA: 2s - loss: 0.0127 - accuracy: 0.9951
 14/375 [>.............................] - ETA: 3s - loss: 0.0121 - accuracy: 0.9961
 21/375 [>.............................] - ETA: 2s - loss: 0.0166 - accuracy: 0.9952
 28/375 [=>............................] - ETA: 2s - loss: 0.0169 - accuracy: 0.9944
 34/375 [=>............................] - ETA: 2s - loss: 0.0167 - accuracy: 0.9947
 40/375 [==>...........................] - ETA: 2s - loss: 0.0156 - accuracy: 0.9953
 46/375 [==>...........................] - ETA: 2s - loss: 0.0170 - accuracy: 0.9949
 52/375 [===>..........................] - ETA: 2s - loss: 0.0168 - accuracy: 0.9949
 59/375 [===>..........................] - ETA: 2s - loss: 0.0167 - accuracy: 0.9951
 65/375 [====>.........................] - ETA: 2s - loss: 0.0165 - accuracy: 0.9952
 71/375 [====>.........................] - ETA: 2s - loss: 0.0158 - accuracy: 0.9955
 77/375 [=====>........................] - ETA: 2s - loss: 0.0156 - accuracy: 0.9954
 83/375 [=====>........................] - ETA: 2s - loss: 0.0151 - accuracy: 0.9957
 89/375 [======>.......................] - ETA: 2s - loss: 0.0152 - accuracy: 0.9958
 95/375 [======>.......................] - ETA: 2s - loss: 0.0149 - accuracy: 0.9961
101/375 [=======>......................] - ETA: 2s - loss: 0.0151 - accuracy: 0.9961
107/375 [=======>......................] - ETA: 2s - loss: 0.0149 - accuracy: 0.9961
112/375 [=======>......................] - ETA: 2s - loss: 0.0155 - accuracy: 0.9960
118/375 [========>.....................] - ETA: 2s - loss: 0.0157 - accuracy: 0.9960
124/375 [========>.....................] - ETA: 2s - loss: 0.0154 - accuracy: 0.9961
129/375 [=========>....................] - ETA: 2s - loss: 0.0151 - accuracy: 0.9962
135/375 [=========>....................] - ETA: 2s - loss: 0.0147 - accuracy: 0.9964
141/375 [==========>...................] - ETA: 2s - loss: 0.0147 - accuracy: 0.9964
147/375 [==========>...................] - ETA: 2s - loss: 0.0150 - accuracy: 0.9961
153/375 [===========>..................] - ETA: 1s - loss: 0.0156 - accuracy: 0.9960
159/375 [===========>..................] - ETA: 1s - loss: 0.0153 - accuracy: 0.9961
165/375 [============>.................] - ETA: 1s - loss: 0.0154 - accuracy: 0.9961
171/375 [============>.................] - ETA: 1s - loss: 0.0154 - accuracy: 0.9961
178/375 [=============>................] - ETA: 1s - loss: 0.0158 - accuracy: 0.9959
184/375 [=============>................] - ETA: 1s - loss: 0.0158 - accuracy: 0.9959
190/375 [==============>...............] - ETA: 1s - loss: 0.0156 - accuracy: 0.9960
196/375 [==============>...............] - ETA: 1s - loss: 0.0157 - accuracy: 0.9960
202/375 [===============>..............] - ETA: 1s - loss: 0.0157 - accuracy: 0.9961
209/375 [===============>..............] - ETA: 1s - loss: 0.0161 - accuracy: 0.9960
215/375 [================>.............] - ETA: 1s - loss: 0.0161 - accuracy: 0.9961
221/375 [================>.............] - ETA: 1s - loss: 0.0161 - accuracy: 0.9961
227/375 [=================>............] - ETA: 1s - loss: 0.0160 - accuracy: 0.9961
234/375 [=================>............] - ETA: 1s - loss: 0.0157 - accuracy: 0.9962
240/375 [==================>...........] - ETA: 1s - loss: 0.0155 - accuracy: 0.9963
247/375 [==================>...........] - ETA: 1s - loss: 0.0155 - accuracy: 0.9963
253/375 [===================>..........] - ETA: 1s - loss: 0.0153 - accuracy: 0.9963
259/375 [===================>..........] - ETA: 1s - loss: 0.0151 - accuracy: 0.9964
265/375 [====================>.........] - ETA: 0s - loss: 0.0151 - accuracy: 0.9964
271/375 [====================>.........] - ETA: 0s - loss: 0.0152 - accuracy: 0.9964
277/375 [=====================>........] - ETA: 0s - loss: 0.0151 - accuracy: 0.9964
283/375 [=====================>........] - ETA: 0s - loss: 0.0153 - accuracy: 0.9964
289/375 [======================>.......] - ETA: 0s - loss: 0.0155 - accuracy: 0.9964
295/375 [======================>.......] - ETA: 0s - loss: 0.0156 - accuracy: 0.9964
302/375 [=======================>......] - ETA: 0s - loss: 0.0155 - accuracy: 0.9964
308/375 [=======================>......] - ETA: 0s - loss: 0.0156 - accuracy: 0.9964
314/375 [========================>.....] - ETA: 0s - loss: 0.0157 - accuracy: 0.9964
320/375 [========================>.....] - ETA: 0s - loss: 0.0157 - accuracy: 0.9964
327/375 [=========================>....] - ETA: 0s - loss: 0.0158 - accuracy: 0.9963
333/375 [=========================>....] - ETA: 0s - loss: 0.0159 - accuracy: 0.9963
339/375 [==========================>...] - ETA: 0s - loss: 0.0159 - accuracy: 0.9963
345/375 [==========================>...] - ETA: 0s - loss: 0.0159 - accuracy: 0.9963
351/375 [===========================>..] - ETA: 0s - loss: 0.0159 - accuracy: 0.9963
357/375 [===========================>..] - ETA: 0s - loss: 0.0159 - accuracy: 0.9963
364/375 [============================>.] - ETA: 0s - loss: 0.0159 - accuracy: 0.9964
371/375 [============================>.] - ETA: 0s - loss: 0.0157 - accuracy: 0.9964
375/375 [==============================] - 3s 9ms/step - loss: 0.0156 - accuracy: 0.9965

375/375 [==============================] - 4s 11ms/step - loss: 0.0156 - accuracy: 0.9965 - val_loss: 0.0757 - val_accuracy: 0.9786 - lr: 1.0000e-03
Epoch 12/20

  1/375 [..............................] - ETA: 2s - loss: 0.0093 - accuracy: 1.0000
  8/375 [..............................] - ETA: 3s - loss: 0.0092 - accuracy: 1.0000
 14/375 [>.............................] - ETA: 3s - loss: 0.0094 - accuracy: 0.9983
 21/375 [>.............................] - ETA: 2s - loss: 0.0149 - accuracy: 0.9981
 28/375 [=>............................] - ETA: 2s - loss: 0.0167 - accuracy: 0.9972
 35/375 [=>............................] - ETA: 2s - loss: 0.0163 - accuracy: 0.9973
 42/375 [==>...........................] - ETA: 2s - loss: 0.0154 - accuracy: 0.9972
 46/375 [==>...........................] - ETA: 2s - loss: 0.0150 - accuracy: 0.9975
 51/375 [===>..........................] - ETA: 2s - loss: 0.0161 - accuracy: 0.9969
 57/375 [===>..........................] - ETA: 2s - loss: 0.0161 - accuracy: 0.9970
 64/375 [====>.........................] - ETA: 2s - loss: 0.0169 - accuracy: 0.9969
 70/375 [====>.........................] - ETA: 2s - loss: 0.0166 - accuracy: 0.9969
 76/375 [=====>........................] - ETA: 2s - loss: 0.0164 - accuracy: 0.9969
 81/375 [=====>........................] - ETA: 2s - loss: 0.0161 - accuracy: 0.9968
 87/375 [=====>........................] - ETA: 2s - loss: 0.0157 - accuracy: 0.9969
 93/375 [======>.......................] - ETA: 2s - loss: 0.0153 - accuracy: 0.9969
 98/375 [======>.......................] - ETA: 2s - loss: 0.0154 - accuracy: 0.9967
104/375 [=======>......................] - ETA: 2s - loss: 0.0153 - accuracy: 0.9968
110/375 [=======>......................] - ETA: 2s - loss: 0.0149 - accuracy: 0.9969
116/375 [========>.....................] - ETA: 2s - loss: 0.0149 - accuracy: 0.9969
121/375 [========>.....................] - ETA: 2s - loss: 0.0150 - accuracy: 0.9970
127/375 [=========>....................] - ETA: 2s - loss: 0.0145 - accuracy: 0.9971
133/375 [=========>....................] - ETA: 2s - loss: 0.0144 - accuracy: 0.9972
139/375 [==========>...................] - ETA: 2s - loss: 0.0142 - accuracy: 0.9973
145/375 [==========>...................] - ETA: 2s - loss: 0.0146 - accuracy: 0.9973
151/375 [===========>..................] - ETA: 2s - loss: 0.0144 - accuracy: 0.9973
157/375 [===========>..................] - ETA: 2s - loss: 0.0146 - accuracy: 0.9973
163/375 [============>.................] - ETA: 1s - loss: 0.0143 - accuracy: 0.9973
169/375 [============>.................] - ETA: 1s - loss: 0.0149 - accuracy: 0.9971
175/375 [=============>................] - ETA: 1s - loss: 0.0148 - accuracy: 0.9971
181/375 [=============>................] - ETA: 1s - loss: 0.0149 - accuracy: 0.9970
187/375 [=============>................] - ETA: 1s - loss: 0.0151 - accuracy: 0.9969
193/375 [==============>...............] - ETA: 1s - loss: 0.0152 - accuracy: 0.9968
199/375 [==============>...............] - ETA: 1s - loss: 0.0150 - accuracy: 0.9969
205/375 [===============>..............] - ETA: 1s - loss: 0.0148 - accuracy: 0.9970
211/375 [===============>..............] - ETA: 1s - loss: 0.0147 - accuracy: 0.9970
215/375 [================>.............] - ETA: 1s - loss: 0.0146 - accuracy: 0.9971
221/375 [================>.............] - ETA: 1s - loss: 0.0146 - accuracy: 0.9970
227/375 [=================>............] - ETA: 1s - loss: 0.0148 - accuracy: 0.9969
232/375 [=================>............] - ETA: 1s - loss: 0.0150 - accuracy: 0.9968
237/375 [=================>............] - ETA: 1s - loss: 0.0152 - accuracy: 0.9967
243/375 [==================>...........] - ETA: 1s - loss: 0.0155 - accuracy: 0.9966
249/375 [==================>...........] - ETA: 1s - loss: 0.0154 - accuracy: 0.9966
255/375 [===================>..........] - ETA: 1s - loss: 0.0156 - accuracy: 0.9966
261/375 [===================>..........] - ETA: 1s - loss: 0.0156 - accuracy: 0.9966
266/375 [====================>.........] - ETA: 1s - loss: 0.0156 - accuracy: 0.9966
271/375 [====================>.........] - ETA: 0s - loss: 0.0155 - accuracy: 0.9966
277/375 [=====================>........] - ETA: 0s - loss: 0.0155 - accuracy: 0.9965
283/375 [=====================>........] - ETA: 0s - loss: 0.0156 - accuracy: 0.9965
290/375 [======================>.......] - ETA: 0s - loss: 0.0156 - accuracy: 0.9965
296/375 [======================>.......] - ETA: 0s - loss: 0.0155 - accuracy: 0.9965
302/375 [=======================>......] - ETA: 0s - loss: 0.0155 - accuracy: 0.9965
308/375 [=======================>......] - ETA: 0s - loss: 0.0154 - accuracy: 0.9965
313/375 [========================>.....] - ETA: 0s - loss: 0.0153 - accuracy: 0.9966
318/375 [========================>.....] - ETA: 0s - loss: 0.0153 - accuracy: 0.9966
324/375 [========================>.....] - ETA: 0s - loss: 0.0153 - accuracy: 0.9966
329/375 [=========================>....] - ETA: 0s - loss: 0.0153 - accuracy: 0.9966
335/375 [=========================>....] - ETA: 0s - loss: 0.0155 - accuracy: 0.9966
341/375 [==========================>...] - ETA: 0s - loss: 0.0154 - accuracy: 0.9966
347/375 [==========================>...] - ETA: 0s - loss: 0.0154 - accuracy: 0.9966
352/375 [===========================>..] - ETA: 0s - loss: 0.0153 - accuracy: 0.9966
358/375 [===========================>..] - ETA: 0s - loss: 0.0153 - accuracy: 0.9966
364/375 [============================>.] - ETA: 0s - loss: 0.0153 - accuracy: 0.9965
369/375 [============================>.] - ETA: 0s - loss: 0.0155 - accuracy: 0.9965
373/375 [============================>.] - ETA: 0s - loss: 0.0154 - accuracy: 0.9965
375/375 [==============================] - 4s 10ms/step - loss: 0.0154 - accuracy: 0.9965

375/375 [==============================] - 5s 12ms/step - loss: 0.0154 - accuracy: 0.9965 - val_loss: 0.0757 - val_accuracy: 0.9785 - lr: 1.0000e-04
Epoch 13/20

  1/375 [..............................] - ETA: 2s - loss: 0.0109 - accuracy: 0.9922
  7/375 [..............................] - ETA: 3s - loss: 0.0141 - accuracy: 0.9955
 13/375 [>.............................] - ETA: 3s - loss: 0.0123 - accuracy: 0.9958
 20/375 [>.............................] - ETA: 3s - loss: 0.0130 - accuracy: 0.9957
 26/375 [=>............................] - ETA: 2s - loss: 0.0132 - accuracy: 0.9958
 30/375 [=>............................] - ETA: 3s - loss: 0.0123 - accuracy: 0.9964
 36/375 [=>............................] - ETA: 3s - loss: 0.0121 - accuracy: 0.9965
 42/375 [==>...........................] - ETA: 3s - loss: 0.0118 - accuracy: 0.9968
 47/375 [==>...........................] - ETA: 3s - loss: 0.0125 - accuracy: 0.9968
 53/375 [===>..........................] - ETA: 3s - loss: 0.0132 - accuracy: 0.9965
 58/375 [===>..........................] - ETA: 3s - loss: 0.0131 - accuracy: 0.9965
 63/375 [====>.........................] - ETA: 3s - loss: 0.0132 - accuracy: 0.9965
 68/375 [====>.........................] - ETA: 2s - loss: 0.0135 - accuracy: 0.9963
 74/375 [====>.........................] - ETA: 2s - loss: 0.0134 - accuracy: 0.9964
 79/375 [=====>........................] - ETA: 2s - loss: 0.0132 - accuracy: 0.9965
 85/375 [=====>........................] - ETA: 2s - loss: 0.0135 - accuracy: 0.9966
 91/375 [======>.......................] - ETA: 2s - loss: 0.0137 - accuracy: 0.9966
 97/375 [======>.......................] - ETA: 2s - loss: 0.0139 - accuracy: 0.9966
103/375 [=======>......................] - ETA: 2s - loss: 0.0145 - accuracy: 0.9965
109/375 [=======>......................] - ETA: 2s - loss: 0.0143 - accuracy: 0.9966
115/375 [========>.....................] - ETA: 2s - loss: 0.0145 - accuracy: 0.9965
121/375 [========>.....................] - ETA: 2s - loss: 0.0141 - accuracy: 0.9967
127/375 [=========>....................] - ETA: 2s - loss: 0.0141 - accuracy: 0.9967
132/375 [=========>....................] - ETA: 2s - loss: 0.0143 - accuracy: 0.9966
137/375 [=========>....................] - ETA: 2s - loss: 0.0145 - accuracy: 0.9966
143/375 [==========>...................] - ETA: 2s - loss: 0.0144 - accuracy: 0.9966
149/375 [==========>...................] - ETA: 2s - loss: 0.0147 - accuracy: 0.9964
154/375 [===========>..................] - ETA: 2s - loss: 0.0145 - accuracy: 0.9964
159/375 [===========>..................] - ETA: 2s - loss: 0.0144 - accuracy: 0.9965
164/375 [============>.................] - ETA: 2s - loss: 0.0145 - accuracy: 0.9964
169/375 [============>.................] - ETA: 1s - loss: 0.0142 - accuracy: 0.9965
174/375 [============>.................] - ETA: 1s - loss: 0.0142 - accuracy: 0.9965
180/375 [=============>................] - ETA: 1s - loss: 0.0145 - accuracy: 0.9965
186/375 [=============>................] - ETA: 1s - loss: 0.0145 - accuracy: 0.9966
192/375 [==============>...............] - ETA: 1s - loss: 0.0145 - accuracy: 0.9966
198/375 [==============>...............] - ETA: 1s - loss: 0.0146 - accuracy: 0.9965
204/375 [===============>..............] - ETA: 1s - loss: 0.0146 - accuracy: 0.9965
211/375 [===============>..............] - ETA: 1s - loss: 0.0145 - accuracy: 0.9966
218/375 [================>.............] - ETA: 1s - loss: 0.0143 - accuracy: 0.9966
223/375 [================>.............] - ETA: 1s - loss: 0.0146 - accuracy: 0.9966
230/375 [=================>............] - ETA: 1s - loss: 0.0147 - accuracy: 0.9966
237/375 [=================>............] - ETA: 1s - loss: 0.0147 - accuracy: 0.9966
244/375 [==================>...........] - ETA: 1s - loss: 0.0147 - accuracy: 0.9966
251/375 [===================>..........] - ETA: 1s - loss: 0.0151 - accuracy: 0.9965
258/375 [===================>..........] - ETA: 1s - loss: 0.0153 - accuracy: 0.9965
265/375 [====================>.........] - ETA: 1s - loss: 0.0155 - accuracy: 0.9964
272/375 [====================>.........] - ETA: 0s - loss: 0.0156 - accuracy: 0.9965
278/375 [=====================>........] - ETA: 0s - loss: 0.0156 - accuracy: 0.9965
284/375 [=====================>........] - ETA: 0s - loss: 0.0155 - accuracy: 0.9965
290/375 [======================>.......] - ETA: 0s - loss: 0.0155 - accuracy: 0.9965
296/375 [======================>.......] - ETA: 0s - loss: 0.0156 - accuracy: 0.9965
303/375 [=======================>......] - ETA: 0s - loss: 0.0154 - accuracy: 0.9965
310/375 [=======================>......] - ETA: 0s - loss: 0.0153 - accuracy: 0.9966
317/375 [========================>.....] - ETA: 0s - loss: 0.0154 - accuracy: 0.9966
324/375 [========================>.....] - ETA: 0s - loss: 0.0153 - accuracy: 0.9966
331/375 [=========================>....] - ETA: 0s - loss: 0.0153 - accuracy: 0.9966
338/375 [==========================>...] - ETA: 0s - loss: 0.0153 - accuracy: 0.9966
345/375 [==========================>...] - ETA: 0s - loss: 0.0152 - accuracy: 0.9966
352/375 [===========================>..] - ETA: 0s - loss: 0.0154 - accuracy: 0.9966
360/375 [===========================>..] - ETA: 0s - loss: 0.0155 - accuracy: 0.9966
367/375 [============================>.] - ETA: 0s - loss: 0.0155 - accuracy: 0.9966
374/375 [============================>.] - ETA: 0s - loss: 0.0154 - accuracy: 0.9966
375/375 [==============================] - 3s 9ms/step - loss: 0.0153 - accuracy: 0.9966

375/375 [==============================] - 4s 11ms/step - loss: 0.0153 - accuracy: 0.9966 - val_loss: 0.0757 - val_accuracy: 0.9785 - lr: 1.0000e-05
Epoch 14/20

  1/375 [..............................] - ETA: 3s - loss: 0.0538 - accuracy: 0.9688
  6/375 [..............................] - ETA: 3s - loss: 0.0217 - accuracy: 0.9909
 12/375 [..............................] - ETA: 3s - loss: 0.0181 - accuracy: 0.9935
 18/375 [>.............................] - ETA: 3s - loss: 0.0174 - accuracy: 0.9944
 25/375 [=>............................] - ETA: 3s - loss: 0.0170 - accuracy: 0.9947
 32/375 [=>............................] - ETA: 3s - loss: 0.0164 - accuracy: 0.9949
 39/375 [==>...........................] - ETA: 2s - loss: 0.0154 - accuracy: 0.9954
 45/375 [==>...........................] - ETA: 2s - loss: 0.0148 - accuracy: 0.9957
 51/375 [===>..........................] - ETA: 2s - loss: 0.0147 - accuracy: 0.9960
 57/375 [===>..........................] - ETA: 2s - loss: 0.0146 - accuracy: 0.9960
 61/375 [===>..........................] - ETA: 2s - loss: 0.0146 - accuracy: 0.9960
 67/375 [====>.........................] - ETA: 2s - loss: 0.0144 - accuracy: 0.9959
 72/375 [====>.........................] - ETA: 2s - loss: 0.0144 - accuracy: 0.9960
 78/375 [=====>........................] - ETA: 2s - loss: 0.0144 - accuracy: 0.9960
 85/375 [=====>........................] - ETA: 2s - loss: 0.0144 - accuracy: 0.9959
 92/375 [======>.......................] - ETA: 2s - loss: 0.0146 - accuracy: 0.9961
 99/375 [======>.......................] - ETA: 2s - loss: 0.0152 - accuracy: 0.9961
106/375 [=======>......................] - ETA: 2s - loss: 0.0149 - accuracy: 0.9962
113/375 [========>.....................] - ETA: 2s - loss: 0.0155 - accuracy: 0.9961
121/375 [========>.....................] - ETA: 2s - loss: 0.0152 - accuracy: 0.9962
128/375 [=========>....................] - ETA: 2s - loss: 0.0153 - accuracy: 0.9962
134/375 [=========>....................] - ETA: 2s - loss: 0.0151 - accuracy: 0.9963
140/375 [==========>...................] - ETA: 2s - loss: 0.0153 - accuracy: 0.9962
145/375 [==========>...................] - ETA: 1s - loss: 0.0154 - accuracy: 0.9962
151/375 [===========>..................] - ETA: 1s - loss: 0.0152 - accuracy: 0.9963
157/375 [===========>..................] - ETA: 1s - loss: 0.0149 - accuracy: 0.9964
164/375 [============>.................] - ETA: 1s - loss: 0.0147 - accuracy: 0.9966
171/375 [============>.................] - ETA: 1s - loss: 0.0145 - accuracy: 0.9966
178/375 [=============>................] - ETA: 1s - loss: 0.0145 - accuracy: 0.9966
185/375 [=============>................] - ETA: 1s - loss: 0.0142 - accuracy: 0.9967
192/375 [==============>...............] - ETA: 1s - loss: 0.0146 - accuracy: 0.9966
199/375 [==============>...............] - ETA: 1s - loss: 0.0146 - accuracy: 0.9967
206/375 [===============>..............] - ETA: 1s - loss: 0.0144 - accuracy: 0.9968
213/375 [================>.............] - ETA: 1s - loss: 0.0143 - accuracy: 0.9968
220/375 [================>.............] - ETA: 1s - loss: 0.0149 - accuracy: 0.9968
227/375 [=================>............] - ETA: 1s - loss: 0.0148 - accuracy: 0.9969
234/375 [=================>............] - ETA: 1s - loss: 0.0149 - accuracy: 0.9969
241/375 [==================>...........] - ETA: 1s - loss: 0.0151 - accuracy: 0.9969
248/375 [==================>...........] - ETA: 1s - loss: 0.0149 - accuracy: 0.9969
255/375 [===================>..........] - ETA: 0s - loss: 0.0151 - accuracy: 0.9968
262/375 [===================>..........] - ETA: 0s - loss: 0.0150 - accuracy: 0.9968
269/375 [====================>.........] - ETA: 0s - loss: 0.0150 - accuracy: 0.9968
276/375 [=====================>........] - ETA: 0s - loss: 0.0149 - accuracy: 0.9968
283/375 [=====================>........] - ETA: 0s - loss: 0.0149 - accuracy: 0.9968
290/375 [======================>.......] - ETA: 0s - loss: 0.0149 - accuracy: 0.9967
297/375 [======================>.......] - ETA: 0s - loss: 0.0150 - accuracy: 0.9967
304/375 [=======================>......] - ETA: 0s - loss: 0.0150 - accuracy: 0.9967
310/375 [=======================>......] - ETA: 0s - loss: 0.0150 - accuracy: 0.9966
317/375 [========================>.....] - ETA: 0s - loss: 0.0150 - accuracy: 0.9966
324/375 [========================>.....] - ETA: 0s - loss: 0.0151 - accuracy: 0.9966
331/375 [=========================>....] - ETA: 0s - loss: 0.0150 - accuracy: 0.9966
338/375 [==========================>...] - ETA: 0s - loss: 0.0150 - accuracy: 0.9966
345/375 [==========================>...] - ETA: 0s - loss: 0.0151 - accuracy: 0.9966
352/375 [===========================>..] - ETA: 0s - loss: 0.0150 - accuracy: 0.9966
359/375 [===========================>..] - ETA: 0s - loss: 0.0150 - accuracy: 0.9966
366/375 [============================>.] - ETA: 0s - loss: 0.0154 - accuracy: 0.9966
373/375 [============================>.] - ETA: 0s - loss: 0.0154 - accuracy: 0.9966
375/375 [==============================] - 3s 8ms/step - loss: 0.0153 - accuracy: 0.9966

375/375 [==============================] - 4s 10ms/step - loss: 0.0153 - accuracy: 0.9966 - val_loss: 0.0757 - val_accuracy: 0.9785 - lr: 1.0000e-06
Epoch 15/20

  1/375 [..............................] - ETA: 2s - loss: 0.0087 - accuracy: 1.0000
  7/375 [..............................] - ETA: 3s - loss: 0.0080 - accuracy: 1.0000
 13/375 [>.............................] - ETA: 3s - loss: 0.0075 - accuracy: 1.0000
 20/375 [>.............................] - ETA: 3s - loss: 0.0100 - accuracy: 0.9988
 26/375 [=>............................] - ETA: 3s - loss: 0.0116 - accuracy: 0.9979
 31/375 [=>............................] - ETA: 3s - loss: 0.0152 - accuracy: 0.9970
 38/375 [==>...........................] - ETA: 2s - loss: 0.0142 - accuracy: 0.9973
 42/375 [==>...........................] - ETA: 3s - loss: 0.0137 - accuracy: 0.9976
 49/375 [==>...........................] - ETA: 2s - loss: 0.0132 - accuracy: 0.9974
 56/375 [===>..........................] - ETA: 2s - loss: 0.0137 - accuracy: 0.9972
 63/375 [====>.........................] - ETA: 2s - loss: 0.0136 - accuracy: 0.9973
 70/375 [====>.........................] - ETA: 2s - loss: 0.0137 - accuracy: 0.9971
 77/375 [=====>........................] - ETA: 2s - loss: 0.0134 - accuracy: 0.9973
 84/375 [=====>........................] - ETA: 2s - loss: 0.0132 - accuracy: 0.9974
 91/375 [======>.......................] - ETA: 2s - loss: 0.0136 - accuracy: 0.9973
 97/375 [======>.......................] - ETA: 2s - loss: 0.0140 - accuracy: 0.9971
102/375 [=======>......................] - ETA: 2s - loss: 0.0145 - accuracy: 0.9969
108/375 [=======>......................] - ETA: 2s - loss: 0.0145 - accuracy: 0.9970
114/375 [========>.....................] - ETA: 2s - loss: 0.0145 - accuracy: 0.9969
120/375 [========>.....................] - ETA: 2s - loss: 0.0154 - accuracy: 0.9966
127/375 [=========>....................] - ETA: 2s - loss: 0.0154 - accuracy: 0.9965
134/375 [=========>....................] - ETA: 2s - loss: 0.0156 - accuracy: 0.9964
141/375 [==========>...................] - ETA: 1s - loss: 0.0156 - accuracy: 0.9964
147/375 [==========>...................] - ETA: 1s - loss: 0.0157 - accuracy: 0.9964
154/375 [===========>..................] - ETA: 1s - loss: 0.0155 - accuracy: 0.9965
161/375 [===========>..................] - ETA: 1s - loss: 0.0155 - accuracy: 0.9963
168/375 [============>.................] - ETA: 1s - loss: 0.0155 - accuracy: 0.9963
175/375 [=============>................] - ETA: 1s - loss: 0.0152 - accuracy: 0.9964
182/375 [=============>................] - ETA: 1s - loss: 0.0157 - accuracy: 0.9964
189/375 [==============>...............] - ETA: 1s - loss: 0.0156 - accuracy: 0.9965
196/375 [==============>...............] - ETA: 1s - loss: 0.0154 - accuracy: 0.9966
203/375 [===============>..............] - ETA: 1s - loss: 0.0152 - accuracy: 0.9967
207/375 [===============>..............] - ETA: 1s - loss: 0.0153 - accuracy: 0.9966
214/375 [================>.............] - ETA: 1s - loss: 0.0152 - accuracy: 0.9967
221/375 [================>.............] - ETA: 1s - loss: 0.0152 - accuracy: 0.9967
228/375 [=================>............] - ETA: 1s - loss: 0.0153 - accuracy: 0.9967
235/375 [=================>............] - ETA: 1s - loss: 0.0154 - accuracy: 0.9967
242/375 [==================>...........] - ETA: 1s - loss: 0.0155 - accuracy: 0.9967
249/375 [==================>...........] - ETA: 1s - loss: 0.0154 - accuracy: 0.9967
256/375 [===================>..........] - ETA: 0s - loss: 0.0154 - accuracy: 0.9967
263/375 [====================>.........] - ETA: 0s - loss: 0.0153 - accuracy: 0.9966
270/375 [====================>.........] - ETA: 0s - loss: 0.0152 - accuracy: 0.9967
276/375 [=====================>........] - ETA: 0s - loss: 0.0152 - accuracy: 0.9967
283/375 [=====================>........] - ETA: 0s - loss: 0.0153 - accuracy: 0.9966
290/375 [======================>.......] - ETA: 0s - loss: 0.0156 - accuracy: 0.9966
297/375 [======================>.......] - ETA: 0s - loss: 0.0156 - accuracy: 0.9965
304/375 [=======================>......] - ETA: 0s - loss: 0.0155 - accuracy: 0.9966
311/375 [=======================>......] - ETA: 0s - loss: 0.0154 - accuracy: 0.9965
318/375 [========================>.....] - ETA: 0s - loss: 0.0154 - accuracy: 0.9965
325/375 [=========================>....] - ETA: 0s - loss: 0.0153 - accuracy: 0.9966
332/375 [=========================>....] - ETA: 0s - loss: 0.0153 - accuracy: 0.9966
335/375 [=========================>....] - ETA: 0s - loss: 0.0152 - accuracy: 0.9966
337/375 [=========================>....] - ETA: 0s - loss: 0.0152 - accuracy: 0.9966
344/375 [==========================>...] - ETA: 0s - loss: 0.0154 - accuracy: 0.9965
351/375 [===========================>..] - ETA: 0s - loss: 0.0154 - accuracy: 0.9966
358/375 [===========================>..] - ETA: 0s - loss: 0.0155 - accuracy: 0.9966
365/375 [============================>.] - ETA: 0s - loss: 0.0155 - accuracy: 0.9965
372/375 [============================>.] - ETA: 0s - loss: 0.0154 - accuracy: 0.9966
375/375 [==============================] - 3s 8ms/step - loss: 0.0153 - accuracy: 0.9966

375/375 [==============================] - 4s 11ms/step - loss: 0.0153 - accuracy: 0.9966 - val_loss: 0.0757 - val_accuracy: 0.9785 - lr: 1.0000e-07
Epoch 16/20

  1/375 [..............................] - ETA: 3s - loss: 0.0139 - accuracy: 1.0000
  8/375 [..............................] - ETA: 2s - loss: 0.0180 - accuracy: 0.9971
 14/375 [>.............................] - ETA: 2s - loss: 0.0156 - accuracy: 0.9972
 20/375 [>.............................] - ETA: 3s - loss: 0.0172 - accuracy: 0.9969
 27/375 [=>............................] - ETA: 2s - loss: 0.0191 - accuracy: 0.9959
 33/375 [=>............................] - ETA: 2s - loss: 0.0201 - accuracy: 0.9957
 39/375 [==>...........................] - ETA: 2s - loss: 0.0188 - accuracy: 0.9960
 45/375 [==>...........................] - ETA: 2s - loss: 0.0174 - accuracy: 0.9965
 51/375 [===>..........................] - ETA: 2s - loss: 0.0167 - accuracy: 0.9966
 55/375 [===>..........................] - ETA: 2s - loss: 0.0165 - accuracy: 0.9966
 61/375 [===>..........................] - ETA: 2s - loss: 0.0165 - accuracy: 0.9963
 67/375 [====>.........................] - ETA: 2s - loss: 0.0160 - accuracy: 0.9964
 73/375 [====>.........................] - ETA: 2s - loss: 0.0157 - accuracy: 0.9964
 77/375 [=====>........................] - ETA: 2s - loss: 0.0152 - accuracy: 0.9966
 83/375 [=====>........................] - ETA: 2s - loss: 0.0156 - accuracy: 0.9966
 89/375 [======>.......................] - ETA: 2s - loss: 0.0156 - accuracy: 0.9966
 95/375 [======>.......................] - ETA: 2s - loss: 0.0153 - accuracy: 0.9966
100/375 [=======>......................] - ETA: 2s - loss: 0.0154 - accuracy: 0.9967
102/375 [=======>......................] - ETA: 2s - loss: 0.0153 - accuracy: 0.9968
106/375 [=======>......................] - ETA: 2s - loss: 0.0153 - accuracy: 0.9968
112/375 [=======>......................] - ETA: 2s - loss: 0.0152 - accuracy: 0.9967
118/375 [========>.....................] - ETA: 2s - loss: 0.0153 - accuracy: 0.9967
124/375 [========>.....................] - ETA: 2s - loss: 0.0150 - accuracy: 0.9968
129/375 [=========>....................] - ETA: 2s - loss: 0.0152 - accuracy: 0.9967
134/375 [=========>....................] - ETA: 2s - loss: 0.0149 - accuracy: 0.9968
140/375 [==========>...................] - ETA: 2s - loss: 0.0149 - accuracy: 0.9968
145/375 [==========>...................] - ETA: 2s - loss: 0.0151 - accuracy: 0.9968
151/375 [===========>..................] - ETA: 2s - loss: 0.0159 - accuracy: 0.9968
157/375 [===========>..................] - ETA: 2s - loss: 0.0158 - accuracy: 0.9969
162/375 [===========>..................] - ETA: 2s - loss: 0.0159 - accuracy: 0.9969
167/375 [============>.................] - ETA: 2s - loss: 0.0160 - accuracy: 0.9968
173/375 [============>.................] - ETA: 2s - loss: 0.0163 - accuracy: 0.9967
179/375 [=============>................] - ETA: 1s - loss: 0.0162 - accuracy: 0.9967
185/375 [=============>................] - ETA: 1s - loss: 0.0161 - accuracy: 0.9967
192/375 [==============>...............] - ETA: 1s - loss: 0.0160 - accuracy: 0.9967
199/375 [==============>...............] - ETA: 1s - loss: 0.0159 - accuracy: 0.9967
206/375 [===============>..............] - ETA: 1s - loss: 0.0156 - accuracy: 0.9968
213/375 [================>.............] - ETA: 1s - loss: 0.0155 - accuracy: 0.9968
220/375 [================>.............] - ETA: 1s - loss: 0.0156 - accuracy: 0.9968
227/375 [=================>............] - ETA: 1s - loss: 0.0156 - accuracy: 0.9968
234/375 [=================>............] - ETA: 1s - loss: 0.0156 - accuracy: 0.9968
241/375 [==================>...........] - ETA: 1s - loss: 0.0158 - accuracy: 0.9967
248/375 [==================>...........] - ETA: 1s - loss: 0.0159 - accuracy: 0.9965
254/375 [===================>..........] - ETA: 1s - loss: 0.0159 - accuracy: 0.9965
260/375 [===================>..........] - ETA: 1s - loss: 0.0158 - accuracy: 0.9965
266/375 [====================>.........] - ETA: 1s - loss: 0.0157 - accuracy: 0.9965
272/375 [====================>.........] - ETA: 0s - loss: 0.0157 - accuracy: 0.9966
279/375 [=====================>........] - ETA: 0s - loss: 0.0158 - accuracy: 0.9965
286/375 [=====================>........] - ETA: 0s - loss: 0.0158 - accuracy: 0.9965
293/375 [======================>.......] - ETA: 0s - loss: 0.0156 - accuracy: 0.9965
300/375 [=======================>......] - ETA: 0s - loss: 0.0155 - accuracy: 0.9966
307/375 [=======================>......] - ETA: 0s - loss: 0.0156 - accuracy: 0.9965
314/375 [========================>.....] - ETA: 0s - loss: 0.0155 - accuracy: 0.9965
321/375 [========================>.....] - ETA: 0s - loss: 0.0153 - accuracy: 0.9965
328/375 [=========================>....] - ETA: 0s - loss: 0.0153 - accuracy: 0.9966
335/375 [=========================>....] - ETA: 0s - loss: 0.0152 - accuracy: 0.9966
342/375 [==========================>...] - ETA: 0s - loss: 0.0153 - accuracy: 0.9966
349/375 [==========================>...] - ETA: 0s - loss: 0.0154 - accuracy: 0.9965
356/375 [===========================>..] - ETA: 0s - loss: 0.0155 - accuracy: 0.9966
363/375 [============================>.] - ETA: 0s - loss: 0.0155 - accuracy: 0.9965
370/375 [============================>.] - ETA: 0s - loss: 0.0153 - accuracy: 0.9966
375/375 [==============================] - 3s 9ms/step - loss: 0.0153 - accuracy: 0.9966

375/375 [==============================] - 4s 11ms/step - loss: 0.0153 - accuracy: 0.9966 - val_loss: 0.0757 - val_accuracy: 0.9785 - lr: 1.0000e-08
Epoch 17/20

  1/375 [..............................] - ETA: 2s - loss: 0.0267 - accuracy: 0.9844
  7/375 [..............................] - ETA: 3s - loss: 0.0237 - accuracy: 0.9933
 13/375 [>.............................] - ETA: 3s - loss: 0.0219 - accuracy: 0.9934
 20/375 [>.............................] - ETA: 3s - loss: 0.0180 - accuracy: 0.9953
 26/375 [=>............................] - ETA: 3s - loss: 0.0162 - accuracy: 0.9961
 33/375 [=>............................] - ETA: 2s - loss: 0.0191 - accuracy: 0.9955
 38/375 [==>...........................] - ETA: 3s - loss: 0.0183 - accuracy: 0.9957
 44/375 [==>...........................] - ETA: 3s - loss: 0.0183 - accuracy: 0.9956
 51/375 [===>..........................] - ETA: 3s - loss: 0.0193 - accuracy: 0.9954
 58/375 [===>..........................] - ETA: 2s - loss: 0.0189 - accuracy: 0.9954
 65/375 [====>.........................] - ETA: 2s - loss: 0.0180 - accuracy: 0.9957
 72/375 [====>.........................] - ETA: 2s - loss: 0.0172 - accuracy: 0.9959
 79/375 [=====>........................] - ETA: 2s - loss: 0.0164 - accuracy: 0.9962
 86/375 [=====>........................] - ETA: 2s - loss: 0.0159 - accuracy: 0.9964
 93/375 [======>.......................] - ETA: 2s - loss: 0.0160 - accuracy: 0.9963
100/375 [=======>......................] - ETA: 2s - loss: 0.0157 - accuracy: 0.9964
106/375 [=======>......................] - ETA: 2s - loss: 0.0157 - accuracy: 0.9965
112/375 [=======>......................] - ETA: 2s - loss: 0.0158 - accuracy: 0.9963
118/375 [========>.....................] - ETA: 2s - loss: 0.0160 - accuracy: 0.9962
125/375 [=========>....................] - ETA: 2s - loss: 0.0158 - accuracy: 0.9963
131/375 [=========>....................] - ETA: 2s - loss: 0.0161 - accuracy: 0.9962
137/375 [=========>....................] - ETA: 2s - loss: 0.0162 - accuracy: 0.9962
144/375 [==========>...................] - ETA: 1s - loss: 0.0160 - accuracy: 0.9962
151/375 [===========>..................] - ETA: 1s - loss: 0.0158 - accuracy: 0.9963
157/375 [===========>..................] - ETA: 1s - loss: 0.0156 - accuracy: 0.9964
163/375 [============>.................] - ETA: 1s - loss: 0.0154 - accuracy: 0.9964
169/375 [============>.................] - ETA: 1s - loss: 0.0158 - accuracy: 0.9963
175/375 [=============>................] - ETA: 1s - loss: 0.0157 - accuracy: 0.9964
181/375 [=============>................] - ETA: 1s - loss: 0.0159 - accuracy: 0.9964
187/375 [=============>................] - ETA: 1s - loss: 0.0156 - accuracy: 0.9965
193/375 [==============>...............] - ETA: 1s - loss: 0.0158 - accuracy: 0.9964
198/375 [==============>...............] - ETA: 1s - loss: 0.0159 - accuracy: 0.9964
204/375 [===============>..............] - ETA: 1s - loss: 0.0157 - accuracy: 0.9964
210/375 [===============>..............] - ETA: 1s - loss: 0.0160 - accuracy: 0.9964
216/375 [================>.............] - ETA: 1s - loss: 0.0162 - accuracy: 0.9964
222/375 [================>.............] - ETA: 1s - loss: 0.0165 - accuracy: 0.9962
228/375 [=================>............] - ETA: 1s - loss: 0.0164 - accuracy: 0.9963
234/375 [=================>............] - ETA: 1s - loss: 0.0162 - accuracy: 0.9964
240/375 [==================>...........] - ETA: 1s - loss: 0.0161 - accuracy: 0.9964
246/375 [==================>...........] - ETA: 1s - loss: 0.0160 - accuracy: 0.9964
252/375 [===================>..........] - ETA: 1s - loss: 0.0158 - accuracy: 0.9965
258/375 [===================>..........] - ETA: 1s - loss: 0.0157 - accuracy: 0.9965
264/375 [====================>.........] - ETA: 0s - loss: 0.0157 - accuracy: 0.9965
270/375 [====================>.........] - ETA: 0s - loss: 0.0157 - accuracy: 0.9965
277/375 [=====================>........] - ETA: 0s - loss: 0.0158 - accuracy: 0.9965
284/375 [=====================>........] - ETA: 0s - loss: 0.0158 - accuracy: 0.9964
291/375 [======================>.......] - ETA: 0s - loss: 0.0158 - accuracy: 0.9964
298/375 [======================>.......] - ETA: 0s - loss: 0.0157 - accuracy: 0.9964
305/375 [=======================>......] - ETA: 0s - loss: 0.0155 - accuracy: 0.9965
312/375 [=======================>......] - ETA: 0s - loss: 0.0156 - accuracy: 0.9965
319/375 [========================>.....] - ETA: 0s - loss: 0.0155 - accuracy: 0.9965
326/375 [=========================>....] - ETA: 0s - loss: 0.0154 - accuracy: 0.9965
333/375 [=========================>....] - ETA: 0s - loss: 0.0155 - accuracy: 0.9965
340/375 [==========================>...] - ETA: 0s - loss: 0.0153 - accuracy: 0.9966
346/375 [==========================>...] - ETA: 0s - loss: 0.0153 - accuracy: 0.9966
353/375 [===========================>..] - ETA: 0s - loss: 0.0154 - accuracy: 0.9966
360/375 [===========================>..] - ETA: 0s - loss: 0.0153 - accuracy: 0.9966
367/375 [============================>.] - ETA: 0s - loss: 0.0154 - accuracy: 0.9966
374/375 [============================>.] - ETA: 0s - loss: 0.0153 - accuracy: 0.9966
375/375 [==============================] - 3s 8ms/step - loss: 0.0153 - accuracy: 0.9966

375/375 [==============================] - 4s 11ms/step - loss: 0.0153 - accuracy: 0.9966 - val_loss: 0.0757 - val_accuracy: 0.9785 - lr: 1.0000e-09
Epoch 18/20

  1/375 [..............................] - ETA: 2s - loss: 0.0354 - accuracy: 0.9844
  8/375 [..............................] - ETA: 2s - loss: 0.0167 - accuracy: 0.9951
 14/375 [>.............................] - ETA: 3s - loss: 0.0166 - accuracy: 0.9967
 20/375 [>.............................] - ETA: 2s - loss: 0.0155 - accuracy: 0.9973
 26/375 [=>............................] - ETA: 2s - loss: 0.0160 - accuracy: 0.9970
 32/375 [=>............................] - ETA: 2s - loss: 0.0152 - accuracy: 0.9976
 36/375 [=>............................] - ETA: 3s - loss: 0.0157 - accuracy: 0.9972
 41/375 [==>...........................] - ETA: 3s - loss: 0.0151 - accuracy: 0.9971
 47/375 [==>...........................] - ETA: 3s - loss: 0.0154 - accuracy: 0.9968
 54/375 [===>..........................] - ETA: 2s - loss: 0.0157 - accuracy: 0.9967
 61/375 [===>..........................] - ETA: 2s - loss: 0.0158 - accuracy: 0.9965
 68/375 [====>.........................] - ETA: 2s - loss: 0.0160 - accuracy: 0.9961
 75/375 [=====>........................] - ETA: 2s - loss: 0.0152 - accuracy: 0.9964
 82/375 [=====>........................] - ETA: 2s - loss: 0.0151 - accuracy: 0.9966
 89/375 [======>.......................] - ETA: 2s - loss: 0.0157 - accuracy: 0.9964
 96/375 [======>.......................] - ETA: 2s - loss: 0.0160 - accuracy: 0.9963
103/375 [=======>......................] - ETA: 2s - loss: 0.0160 - accuracy: 0.9964
109/375 [=======>......................] - ETA: 2s - loss: 0.0156 - accuracy: 0.9966
115/375 [========>.....................] - ETA: 2s - loss: 0.0155 - accuracy: 0.9966
121/375 [========>.....................] - ETA: 2s - loss: 0.0155 - accuracy: 0.9966
128/375 [=========>....................] - ETA: 2s - loss: 0.0156 - accuracy: 0.9964
134/375 [=========>....................] - ETA: 2s - loss: 0.0154 - accuracy: 0.9964
141/375 [==========>...................] - ETA: 2s - loss: 0.0151 - accuracy: 0.9965
148/375 [==========>...................] - ETA: 1s - loss: 0.0152 - accuracy: 0.9965
155/375 [===========>..................] - ETA: 1s - loss: 0.0155 - accuracy: 0.9965
161/375 [===========>..................] - ETA: 1s - loss: 0.0155 - accuracy: 0.9965
167/375 [============>.................] - ETA: 1s - loss: 0.0154 - accuracy: 0.9964
173/375 [============>.................] - ETA: 1s - loss: 0.0152 - accuracy: 0.9965
179/375 [=============>................] - ETA: 1s - loss: 0.0151 - accuracy: 0.9964
185/375 [=============>................] - ETA: 1s - loss: 0.0150 - accuracy: 0.9965
191/375 [==============>...............] - ETA: 1s - loss: 0.0150 - accuracy: 0.9965
197/375 [==============>...............] - ETA: 1s - loss: 0.0149 - accuracy: 0.9966
203/375 [===============>..............] - ETA: 1s - loss: 0.0149 - accuracy: 0.9965
209/375 [===============>..............] - ETA: 1s - loss: 0.0149 - accuracy: 0.9965
215/375 [================>.............] - ETA: 1s - loss: 0.0148 - accuracy: 0.9965
221/375 [================>.............] - ETA: 1s - loss: 0.0148 - accuracy: 0.9966
227/375 [=================>............] - ETA: 1s - loss: 0.0146 - accuracy: 0.9967
234/375 [=================>............] - ETA: 1s - loss: 0.0147 - accuracy: 0.9967
241/375 [==================>...........] - ETA: 1s - loss: 0.0149 - accuracy: 0.9967
248/375 [==================>...........] - ETA: 1s - loss: 0.0148 - accuracy: 0.9967
255/375 [===================>..........] - ETA: 1s - loss: 0.0150 - accuracy: 0.9966
262/375 [===================>..........] - ETA: 0s - loss: 0.0149 - accuracy: 0.9966
269/375 [====================>.........] - ETA: 0s - loss: 0.0147 - accuracy: 0.9967
276/375 [=====================>........] - ETA: 0s - loss: 0.0149 - accuracy: 0.9967
283/375 [=====================>........] - ETA: 0s - loss: 0.0152 - accuracy: 0.9966
290/375 [======================>.......] - ETA: 0s - loss: 0.0151 - accuracy: 0.9966
297/375 [======================>.......] - ETA: 0s - loss: 0.0151 - accuracy: 0.9966
303/375 [=======================>......] - ETA: 0s - loss: 0.0150 - accuracy: 0.9966
309/375 [=======================>......] - ETA: 0s - loss: 0.0151 - accuracy: 0.9966
315/375 [========================>.....] - ETA: 0s - loss: 0.0150 - accuracy: 0.9966
321/375 [========================>.....] - ETA: 0s - loss: 0.0150 - accuracy: 0.9966
326/375 [=========================>....] - ETA: 0s - loss: 0.0150 - accuracy: 0.9965
333/375 [=========================>....] - ETA: 0s - loss: 0.0149 - accuracy: 0.9966
340/375 [==========================>...] - ETA: 0s - loss: 0.0151 - accuracy: 0.9966
347/375 [==========================>...] - ETA: 0s - loss: 0.0155 - accuracy: 0.9965
354/375 [===========================>..] - ETA: 0s - loss: 0.0155 - accuracy: 0.9965
361/375 [===========================>..] - ETA: 0s - loss: 0.0154 - accuracy: 0.9966
368/375 [============================>.] - ETA: 0s - loss: 0.0154 - accuracy: 0.9966
375/375 [==============================] - 3s 8ms/step - loss: 0.0153 - accuracy: 0.9966

375/375 [==============================] - 4s 11ms/step - loss: 0.0153 - accuracy: 0.9966 - val_loss: 0.0757 - val_accuracy: 0.9785 - lr: 1.0000e-10
Epoch 19/20

  1/375 [..............................] - ETA: 3s - loss: 0.0127 - accuracy: 0.9922
  7/375 [..............................] - ETA: 3s - loss: 0.0087 - accuracy: 0.9967
 14/375 [>.............................] - ETA: 3s - loss: 0.0107 - accuracy: 0.9950
 20/375 [>.............................] - ETA: 3s - loss: 0.0131 - accuracy: 0.9945
 26/375 [=>............................] - ETA: 3s - loss: 0.0130 - accuracy: 0.9952
 32/375 [=>............................] - ETA: 2s - loss: 0.0138 - accuracy: 0.9954
 39/375 [==>...........................] - ETA: 2s - loss: 0.0129 - accuracy: 0.9960
 45/375 [==>...........................] - ETA: 2s - loss: 0.0146 - accuracy: 0.9957
 51/375 [===>..........................] - ETA: 2s - loss: 0.0138 - accuracy: 0.9960
 57/375 [===>..........................] - ETA: 2s - loss: 0.0136 - accuracy: 0.9963
 62/375 [===>..........................] - ETA: 2s - loss: 0.0140 - accuracy: 0.9962
 68/375 [====>.........................] - ETA: 2s - loss: 0.0136 - accuracy: 0.9964
 73/375 [====>.........................] - ETA: 2s - loss: 0.0142 - accuracy: 0.9961
 79/375 [=====>........................] - ETA: 2s - loss: 0.0148 - accuracy: 0.9959
 85/375 [=====>........................] - ETA: 2s - loss: 0.0145 - accuracy: 0.9960
 91/375 [======>.......................] - ETA: 2s - loss: 0.0152 - accuracy: 0.9960
 96/375 [======>.......................] - ETA: 2s - loss: 0.0149 - accuracy: 0.9962
101/375 [=======>......................] - ETA: 2s - loss: 0.0147 - accuracy: 0.9962
107/375 [=======>......................] - ETA: 2s - loss: 0.0143 - accuracy: 0.9964
110/375 [=======>......................] - ETA: 2s - loss: 0.0146 - accuracy: 0.9963
116/375 [========>.....................] - ETA: 2s - loss: 0.0143 - accuracy: 0.9964
122/375 [========>.....................] - ETA: 2s - loss: 0.0141 - accuracy: 0.9965
127/375 [=========>....................] - ETA: 2s - loss: 0.0142 - accuracy: 0.9965
133/375 [=========>....................] - ETA: 2s - loss: 0.0146 - accuracy: 0.9964
138/375 [==========>...................] - ETA: 2s - loss: 0.0149 - accuracy: 0.9964
144/375 [==========>...................] - ETA: 2s - loss: 0.0147 - accuracy: 0.9965
149/375 [==========>...................] - ETA: 2s - loss: 0.0145 - accuracy: 0.9965
154/375 [===========>..................] - ETA: 2s - loss: 0.0146 - accuracy: 0.9965
161/375 [===========>..................] - ETA: 2s - loss: 0.0149 - accuracy: 0.9965
166/375 [============>.................] - ETA: 2s - loss: 0.0151 - accuracy: 0.9965
172/375 [============>.................] - ETA: 1s - loss: 0.0150 - accuracy: 0.9965
177/375 [=============>................] - ETA: 1s - loss: 0.0150 - accuracy: 0.9966
182/375 [=============>................] - ETA: 1s - loss: 0.0150 - accuracy: 0.9965
189/375 [==============>...............] - ETA: 1s - loss: 0.0151 - accuracy: 0.9965
196/375 [==============>...............] - ETA: 1s - loss: 0.0153 - accuracy: 0.9964
203/375 [===============>..............] - ETA: 1s - loss: 0.0152 - accuracy: 0.9965
210/375 [===============>..............] - ETA: 1s - loss: 0.0155 - accuracy: 0.9964
217/375 [================>.............] - ETA: 1s - loss: 0.0155 - accuracy: 0.9964
224/375 [================>.............] - ETA: 1s - loss: 0.0159 - accuracy: 0.9963
231/375 [=================>............] - ETA: 1s - loss: 0.0157 - accuracy: 0.9964
238/375 [==================>...........] - ETA: 1s - loss: 0.0157 - accuracy: 0.9964
245/375 [==================>...........] - ETA: 1s - loss: 0.0157 - accuracy: 0.9965
252/375 [===================>..........] - ETA: 1s - loss: 0.0160 - accuracy: 0.9965
258/375 [===================>..........] - ETA: 1s - loss: 0.0159 - accuracy: 0.9965
264/375 [====================>.........] - ETA: 1s - loss: 0.0158 - accuracy: 0.9965
270/375 [====================>.........] - ETA: 0s - loss: 0.0157 - accuracy: 0.9965
276/375 [=====================>........] - ETA: 0s - loss: 0.0156 - accuracy: 0.9965
282/375 [=====================>........] - ETA: 0s - loss: 0.0157 - accuracy: 0.9965
288/375 [======================>.......] - ETA: 0s - loss: 0.0157 - accuracy: 0.9966
294/375 [======================>.......] - ETA: 0s - loss: 0.0156 - accuracy: 0.9967
300/375 [=======================>......] - ETA: 0s - loss: 0.0154 - accuracy: 0.9967
307/375 [=======================>......] - ETA: 0s - loss: 0.0155 - accuracy: 0.9966
313/375 [========================>.....] - ETA: 0s - loss: 0.0153 - accuracy: 0.9967
319/375 [========================>.....] - ETA: 0s - loss: 0.0152 - accuracy: 0.9967
325/375 [=========================>....] - ETA: 0s - loss: 0.0155 - accuracy: 0.9966
331/375 [=========================>....] - ETA: 0s - loss: 0.0154 - accuracy: 0.9966
337/375 [=========================>....] - ETA: 0s - loss: 0.0154 - accuracy: 0.9967
343/375 [==========================>...] - ETA: 0s - loss: 0.0154 - accuracy: 0.9967
349/375 [==========================>...] - ETA: 0s - loss: 0.0154 - accuracy: 0.9966
355/375 [===========================>..] - ETA: 0s - loss: 0.0154 - accuracy: 0.9966
361/375 [===========================>..] - ETA: 0s - loss: 0.0154 - accuracy: 0.9966
367/375 [============================>.] - ETA: 0s - loss: 0.0153 - accuracy: 0.9966
374/375 [============================>.] - ETA: 0s - loss: 0.0154 - accuracy: 0.9966
375/375 [==============================] - 3s 9ms/step - loss: 0.0153 - accuracy: 0.9966

375/375 [==============================] - 4s 12ms/step - loss: 0.0153 - accuracy: 0.9966 - val_loss: 0.0757 - val_accuracy: 0.9785 - lr: 1.0000e-11
Epoch 20/20

  1/375 [..............................] - ETA: 3s - loss: 0.0097 - accuracy: 1.0000
  7/375 [..............................] - ETA: 3s - loss: 0.0251 - accuracy: 0.9933
 13/375 [>.............................] - ETA: 3s - loss: 0.0255 - accuracy: 0.9928
 18/375 [>.............................] - ETA: 3s - loss: 0.0232 - accuracy: 0.9939
 24/375 [>.............................] - ETA: 3s - loss: 0.0215 - accuracy: 0.9941
 31/375 [=>............................] - ETA: 3s - loss: 0.0195 - accuracy: 0.9950
 36/375 [=>............................] - ETA: 3s - loss: 0.0182 - accuracy: 0.9954
 40/375 [==>...........................] - ETA: 3s - loss: 0.0174 - accuracy: 0.9957
 46/375 [==>...........................] - ETA: 3s - loss: 0.0178 - accuracy: 0.9956
 52/375 [===>..........................] - ETA: 3s - loss: 0.0182 - accuracy: 0.9955
 58/375 [===>..........................] - ETA: 3s - loss: 0.0174 - accuracy: 0.9958
 64/375 [====>.........................] - ETA: 3s - loss: 0.0187 - accuracy: 0.9958
 70/375 [====>.........................] - ETA: 2s - loss: 0.0185 - accuracy: 0.9959
 76/375 [=====>........................] - ETA: 2s - loss: 0.0180 - accuracy: 0.9961
 81/375 [=====>........................] - ETA: 2s - loss: 0.0175 - accuracy: 0.9962
 87/375 [=====>........................] - ETA: 2s - loss: 0.0173 - accuracy: 0.9963
 93/375 [======>.......................] - ETA: 2s - loss: 0.0168 - accuracy: 0.9965
 98/375 [======>.......................] - ETA: 2s - loss: 0.0166 - accuracy: 0.9965
103/375 [=======>......................] - ETA: 2s - loss: 0.0171 - accuracy: 0.9961
108/375 [=======>......................] - ETA: 2s - loss: 0.0173 - accuracy: 0.9961
114/375 [========>.....................] - ETA: 2s - loss: 0.0170 - accuracy: 0.9962
119/375 [========>.....................] - ETA: 2s - loss: 0.0170 - accuracy: 0.9962
124/375 [========>.....................] - ETA: 2s - loss: 0.0172 - accuracy: 0.9961
129/375 [=========>....................] - ETA: 2s - loss: 0.0168 - accuracy: 0.9962
134/375 [=========>....................] - ETA: 2s - loss: 0.0167 - accuracy: 0.9963
139/375 [==========>...................] - ETA: 2s - loss: 0.0165 - accuracy: 0.9963
144/375 [==========>...................] - ETA: 2s - loss: 0.0164 - accuracy: 0.9964
150/375 [===========>..................] - ETA: 2s - loss: 0.0164 - accuracy: 0.9964
155/375 [===========>..................] - ETA: 2s - loss: 0.0161 - accuracy: 0.9965
161/375 [===========>..................] - ETA: 2s - loss: 0.0162 - accuracy: 0.9965
167/375 [============>.................] - ETA: 2s - loss: 0.0159 - accuracy: 0.9966
173/375 [============>.................] - ETA: 1s - loss: 0.0159 - accuracy: 0.9966
178/375 [=============>................] - ETA: 1s - loss: 0.0162 - accuracy: 0.9965
183/375 [=============>................] - ETA: 1s - loss: 0.0164 - accuracy: 0.9965
188/375 [==============>...............] - ETA: 1s - loss: 0.0163 - accuracy: 0.9966
194/375 [==============>...............] - ETA: 1s - loss: 0.0161 - accuracy: 0.9966
197/375 [==============>...............] - ETA: 1s - loss: 0.0161 - accuracy: 0.9965
202/375 [===============>..............] - ETA: 1s - loss: 0.0162 - accuracy: 0.9966
208/375 [===============>..............] - ETA: 1s - loss: 0.0161 - accuracy: 0.9966
213/375 [================>.............] - ETA: 1s - loss: 0.0160 - accuracy: 0.9966
218/375 [================>.............] - ETA: 1s - loss: 0.0159 - accuracy: 0.9966
223/375 [================>.............] - ETA: 1s - loss: 0.0158 - accuracy: 0.9966
229/375 [=================>............] - ETA: 1s - loss: 0.0156 - accuracy: 0.9967
235/375 [=================>............] - ETA: 1s - loss: 0.0154 - accuracy: 0.9968
240/375 [==================>...........] - ETA: 1s - loss: 0.0159 - accuracy: 0.9967
246/375 [==================>...........] - ETA: 1s - loss: 0.0160 - accuracy: 0.9966
251/375 [===================>..........] - ETA: 1s - loss: 0.0158 - accuracy: 0.9967
257/375 [===================>..........] - ETA: 1s - loss: 0.0160 - accuracy: 0.9966
264/375 [====================>.........] - ETA: 1s - loss: 0.0160 - accuracy: 0.9965
271/375 [====================>.........] - ETA: 1s - loss: 0.0161 - accuracy: 0.9965
278/375 [=====================>........] - ETA: 0s - loss: 0.0161 - accuracy: 0.9965
284/375 [=====================>........] - ETA: 0s - loss: 0.0160 - accuracy: 0.9965
288/375 [======================>.......] - ETA: 0s - loss: 0.0161 - accuracy: 0.9965
294/375 [======================>.......] - ETA: 0s - loss: 0.0160 - accuracy: 0.9965
300/375 [=======================>......] - ETA: 0s - loss: 0.0160 - accuracy: 0.9965
306/375 [=======================>......] - ETA: 0s - loss: 0.0158 - accuracy: 0.9965
311/375 [=======================>......] - ETA: 0s - loss: 0.0158 - accuracy: 0.9965
317/375 [========================>.....] - ETA: 0s - loss: 0.0156 - accuracy: 0.9965
323/375 [========================>.....] - ETA: 0s - loss: 0.0155 - accuracy: 0.9966
328/375 [=========================>....] - ETA: 0s - loss: 0.0156 - accuracy: 0.9965
334/375 [=========================>....] - ETA: 0s - loss: 0.0155 - accuracy: 0.9966
339/375 [==========================>...] - ETA: 0s - loss: 0.0154 - accuracy: 0.9966
344/375 [==========================>...] - ETA: 0s - loss: 0.0154 - accuracy: 0.9966
350/375 [===========================>..] - ETA: 0s - loss: 0.0153 - accuracy: 0.9966
355/375 [===========================>..] - ETA: 0s - loss: 0.0154 - accuracy: 0.9965
360/375 [===========================>..] - ETA: 0s - loss: 0.0154 - accuracy: 0.9966
366/375 [============================>.] - ETA: 0s - loss: 0.0155 - accuracy: 0.9965
372/375 [============================>.] - ETA: 0s - loss: 0.0154 - accuracy: 0.9966
375/375 [==============================] - 4s 10ms/step - loss: 0.0153 - accuracy: 0.9966

375/375 [==============================] - 5s 12ms/step - loss: 0.0153 - accuracy: 0.9966 - val_loss: 0.0757 - val_accuracy: 0.9785 - lr: 1.0000e-12

model_2_layer_v2 <- keras_model_sequential() %>%
  layer_dense(units = 32, activation = 'relu', input_shape = 784) %>%
  layer_dense(units = 64, activation = 'relu') %>%
  layer_dense(units = 128, activation = 'relu') %>%
  layer_dense(units = 10, activation = 'softmax')

summary(model_2_layer_v2)

Model: "sequential_8"
___________________________________________________________________________________________________________
 Layer (type)                                   Output Shape                              Param #          
===========================================================================================================
 dense_22 (Dense)                               (None, 32)                                25120            
 dense_21 (Dense)                               (None, 64)                                2112             
 dense_20 (Dense)                               (None, 128)                               8320             
 dense_19 (Dense)                               (None, 10)                                1290             
===========================================================================================================
Total params: 36,842
Trainable params: 36,842
Non-trainable params: 0
___________________________________________________________________________________________________________

model_2_layer_v2 %>% compile(
  loss = "categorical_crossentropy",
  optimizer = optimizer_sgd(lr = 0.1, momentum = 0.9),
  metrics = "accuracy"
)

Warning: the `lr` argument has been renamed to `learning_rate`.

history <- model_2_layer_v2 %>% fit(
  train_images, train_labels,
  validation_split = 0.2,
  batch_size = 128,
  epochs = 20,
  callback = list(
    # callback_early_stopping(patience = 3, restore_best_weights = TRUE, min_delta = 0.0001),
    callback_reduce_lr_on_plateau(patience = 1, factor = 0.1)
    )
  )

Epoch 1/20

2022-08-04 15:40:46.277844: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:113] Plugin optimizer for device_type GPU is enabled.

  1/375 [..............................] - ETA: 2:10 - loss: 2.3363 - accuracy: 0.0703
  5/375 [..............................] - ETA: 4s - loss: 2.3020 - accuracy: 0.1281  
 11/375 [..............................] - ETA: 3s - loss: 2.2104 - accuracy: 0.2585
 17/375 [>.............................] - ETA: 3s - loss: 1.9456 - accuracy: 0.3525
 23/375 [>.............................] - ETA: 3s - loss: 1.7642 - accuracy: 0.4151
 29/375 [=>............................] - ETA: 3s - loss: 1.6795 - accuracy: 0.4534
 35/375 [=>............................] - ETA: 3s - loss: 1.5655 - accuracy: 0.4839
 41/375 [==>...........................] - ETA: 3s - loss: 1.4439 - accuracy: 0.5252
 47/375 [==>...........................] - ETA: 3s - loss: 1.3493 - accuracy: 0.5588
 53/375 [===>..........................] - ETA: 3s - loss: 1.2757 - accuracy: 0.5856
 59/375 [===>..........................] - ETA: 3s - loss: 1.1976 - accuracy: 0.6122
 64/375 [====>.........................] - ETA: 3s - loss: 1.1398 - accuracy: 0.6313
 69/375 [====>.........................] - ETA: 3s - loss: 1.0888 - accuracy: 0.6496
 74/375 [====>.........................] - ETA: 2s - loss: 1.0452 - accuracy: 0.6649
 80/375 [=====>........................] - ETA: 2s - loss: 1.0033 - accuracy: 0.6799
 86/375 [=====>........................] - ETA: 2s - loss: 0.9612 - accuracy: 0.6941
 89/375 [======>.......................] - ETA: 2s - loss: 0.9445 - accuracy: 0.7000
 92/375 [======>.......................] - ETA: 2s - loss: 0.9288 - accuracy: 0.7055
 97/375 [======>.......................] - ETA: 2s - loss: 0.8997 - accuracy: 0.7147
103/375 [=======>......................] - ETA: 2s - loss: 0.8666 - accuracy: 0.7257
108/375 [=======>......................] - ETA: 2s - loss: 0.8424 - accuracy: 0.7339
114/375 [========>.....................] - ETA: 2s - loss: 0.8142 - accuracy: 0.7430
119/375 [========>.....................] - ETA: 2s - loss: 0.7956 - accuracy: 0.7493
124/375 [========>.....................] - ETA: 2s - loss: 0.7768 - accuracy: 0.7549
127/375 [=========>....................] - ETA: 2s - loss: 0.7652 - accuracy: 0.7586
132/375 [=========>....................] - ETA: 2s - loss: 0.7499 - accuracy: 0.7641
138/375 [==========>...................] - ETA: 2s - loss: 0.7295 - accuracy: 0.7707
144/375 [==========>...................] - ETA: 2s - loss: 0.7115 - accuracy: 0.7767
150/375 [===========>..................] - ETA: 2s - loss: 0.6973 - accuracy: 0.7821
156/375 [===========>..................] - ETA: 2s - loss: 0.6825 - accuracy: 0.7867
162/375 [===========>..................] - ETA: 2s - loss: 0.6704 - accuracy: 0.7909
168/375 [============>.................] - ETA: 2s - loss: 0.6576 - accuracy: 0.7951
174/375 [============>.................] - ETA: 2s - loss: 0.6468 - accuracy: 0.7987
180/375 [=============>................] - ETA: 1s - loss: 0.6359 - accuracy: 0.8023
186/375 [=============>................] - ETA: 1s - loss: 0.6261 - accuracy: 0.8057
193/375 [==============>...............] - ETA: 1s - loss: 0.6126 - accuracy: 0.8097
199/375 [==============>...............] - ETA: 1s - loss: 0.6010 - accuracy: 0.8131
205/375 [===============>..............] - ETA: 1s - loss: 0.5894 - accuracy: 0.8169
211/375 [===============>..............] - ETA: 1s - loss: 0.5794 - accuracy: 0.8202
217/375 [================>.............] - ETA: 1s - loss: 0.5714 - accuracy: 0.8229
223/375 [================>.............] - ETA: 1s - loss: 0.5639 - accuracy: 0.8253
229/375 [=================>............] - ETA: 1s - loss: 0.5562 - accuracy: 0.8279
235/375 [=================>............] - ETA: 1s - loss: 0.5494 - accuracy: 0.8298
242/375 [==================>...........] - ETA: 1s - loss: 0.5422 - accuracy: 0.8323
246/375 [==================>...........] - ETA: 1s - loss: 0.5382 - accuracy: 0.8335
252/375 [===================>..........] - ETA: 1s - loss: 0.5312 - accuracy: 0.8354
258/375 [===================>..........] - ETA: 1s - loss: 0.5244 - accuracy: 0.8376
264/375 [====================>.........] - ETA: 1s - loss: 0.5181 - accuracy: 0.8399
270/375 [====================>.........] - ETA: 1s - loss: 0.5122 - accuracy: 0.8418
276/375 [=====================>........] - ETA: 0s - loss: 0.5065 - accuracy: 0.8439
282/375 [=====================>........] - ETA: 0s - loss: 0.5009 - accuracy: 0.8455
288/375 [======================>.......] - ETA: 0s - loss: 0.4962 - accuracy: 0.8470
295/375 [======================>.......] - ETA: 0s - loss: 0.4900 - accuracy: 0.8491
301/375 [=======================>......] - ETA: 0s - loss: 0.4847 - accuracy: 0.8507
307/375 [=======================>......] - ETA: 0s - loss: 0.4811 - accuracy: 0.8520
314/375 [========================>.....] - ETA: 0s - loss: 0.4747 - accuracy: 0.8541
320/375 [========================>.....] - ETA: 0s - loss: 0.4697 - accuracy: 0.8556
326/375 [=========================>....] - ETA: 0s - loss: 0.4669 - accuracy: 0.8566
333/375 [=========================>....] - ETA: 0s - loss: 0.4633 - accuracy: 0.8579
339/375 [==========================>...] - ETA: 0s - loss: 0.4594 - accuracy: 0.8593
344/375 [==========================>...] - ETA: 0s - loss: 0.4565 - accuracy: 0.8602
350/375 [===========================>..] - ETA: 0s - loss: 0.4523 - accuracy: 0.8614
356/375 [===========================>..] - ETA: 0s - loss: 0.4482 - accuracy: 0.8627
362/375 [===========================>..] - ETA: 0s - loss: 0.4443 - accuracy: 0.8640
368/375 [============================>.] - ETA: 0s - loss: 0.4412 - accuracy: 0.8649
374/375 [============================>.] - ETA: 0s - loss: 0.4375 - accuracy: 0.8662
375/375 [==============================] - 4s 10ms/step - loss: 0.4372 - accuracy: 0.8664

2022-08-04 15:40:49.997279: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:113] Plugin optimizer for device_type GPU is enabled.


375/375 [==============================] - 5s 14ms/step - loss: 0.4372 - accuracy: 0.8664 - val_loss: 0.2393 - val_accuracy: 0.9275 - lr: 0.1000
Epoch 2/20

  1/375 [..............................] - ETA: 2s - loss: 0.1909 - accuracy: 0.9297
  6/375 [..............................] - ETA: 5s - loss: 0.2029 - accuracy: 0.9349
 10/375 [..............................] - ETA: 5s - loss: 0.1840 - accuracy: 0.9398
 16/375 [>.............................] - ETA: 4s - loss: 0.1898 - accuracy: 0.9399
 23/375 [>.............................] - ETA: 3s - loss: 0.1958 - accuracy: 0.9416
 30/375 [=>............................] - ETA: 3s - loss: 0.2022 - accuracy: 0.9393
 36/375 [=>............................] - ETA: 3s - loss: 0.2002 - accuracy: 0.9405
 42/375 [==>...........................] - ETA: 3s - loss: 0.2020 - accuracy: 0.9405
 48/375 [==>...........................] - ETA: 3s - loss: 0.1920 - accuracy: 0.9437
 54/375 [===>..........................] - ETA: 3s - loss: 0.1959 - accuracy: 0.9426
 60/375 [===>..........................] - ETA: 2s - loss: 0.1982 - accuracy: 0.9421
 67/375 [====>.........................] - ETA: 2s - loss: 0.2030 - accuracy: 0.9412
 74/375 [====>.........................] - ETA: 2s - loss: 0.2002 - accuracy: 0.9417
 81/375 [=====>........................] - ETA: 2s - loss: 0.2019 - accuracy: 0.9418
 88/375 [======>.......................] - ETA: 2s - loss: 0.2057 - accuracy: 0.9402
 95/375 [======>.......................] - ETA: 2s - loss: 0.2071 - accuracy: 0.9399
102/375 [=======>......................] - ETA: 2s - loss: 0.2060 - accuracy: 0.9399
109/375 [=======>......................] - ETA: 2s - loss: 0.2066 - accuracy: 0.9402
115/375 [========>.....................] - ETA: 2s - loss: 0.2045 - accuracy: 0.9405
121/375 [========>.....................] - ETA: 2s - loss: 0.2020 - accuracy: 0.9413
127/375 [=========>....................] - ETA: 2s - loss: 0.2023 - accuracy: 0.9413
133/375 [=========>....................] - ETA: 2s - loss: 0.2025 - accuracy: 0.9415
140/375 [==========>...................] - ETA: 2s - loss: 0.2015 - accuracy: 0.9420
147/375 [==========>...................] - ETA: 1s - loss: 0.2020 - accuracy: 0.9417
154/375 [===========>..................] - ETA: 1s - loss: 0.2020 - accuracy: 0.9415
161/375 [===========>..................] - ETA: 1s - loss: 0.2018 - accuracy: 0.9418
168/375 [============>.................] - ETA: 1s - loss: 0.2007 - accuracy: 0.9422
175/375 [=============>................] - ETA: 1s - loss: 0.2015 - accuracy: 0.9423
182/375 [=============>................] - ETA: 1s - loss: 0.2009 - accuracy: 0.9425
189/375 [==============>...............] - ETA: 1s - loss: 0.1992 - accuracy: 0.9430
196/375 [==============>...............] - ETA: 1s - loss: 0.1978 - accuracy: 0.9435
203/375 [===============>..............] - ETA: 1s - loss: 0.1966 - accuracy: 0.9440
210/375 [===============>..............] - ETA: 1s - loss: 0.1942 - accuracy: 0.9447
217/375 [================>.............] - ETA: 1s - loss: 0.1927 - accuracy: 0.9452
224/375 [================>.............] - ETA: 1s - loss: 0.1911 - accuracy: 0.9455
231/375 [=================>............] - ETA: 1s - loss: 0.1895 - accuracy: 0.9459
237/375 [=================>............] - ETA: 1s - loss: 0.1893 - accuracy: 0.9460
243/375 [==================>...........] - ETA: 1s - loss: 0.1885 - accuracy: 0.9461
250/375 [===================>..........] - ETA: 1s - loss: 0.1883 - accuracy: 0.9462
257/375 [===================>..........] - ETA: 0s - loss: 0.1883 - accuracy: 0.9463
262/375 [===================>..........] - ETA: 0s - loss: 0.1881 - accuracy: 0.9463
268/375 [====================>.........] - ETA: 0s - loss: 0.1887 - accuracy: 0.9462
274/375 [====================>.........] - ETA: 0s - loss: 0.1888 - accuracy: 0.9461
280/375 [=====================>........] - ETA: 0s - loss: 0.1895 - accuracy: 0.9458
286/375 [=====================>........] - ETA: 0s - loss: 0.1897 - accuracy: 0.9458
292/375 [======================>.......] - ETA: 0s - loss: 0.1894 - accuracy: 0.9457
298/375 [======================>.......] - ETA: 0s - loss: 0.1893 - accuracy: 0.9455
304/375 [=======================>......] - ETA: 0s - loss: 0.1886 - accuracy: 0.9457
310/375 [=======================>......] - ETA: 0s - loss: 0.1884 - accuracy: 0.9458
316/375 [========================>.....] - ETA: 0s - loss: 0.1885 - accuracy: 0.9457
321/375 [========================>.....] - ETA: 0s - loss: 0.1874 - accuracy: 0.9460
326/375 [=========================>....] - ETA: 0s - loss: 0.1877 - accuracy: 0.9459
331/375 [=========================>....] - ETA: 0s - loss: 0.1874 - accuracy: 0.9459
337/375 [=========================>....] - ETA: 0s - loss: 0.1874 - accuracy: 0.9455
344/375 [==========================>...] - ETA: 0s - loss: 0.1868 - accuracy: 0.9456
350/375 [===========================>..] - ETA: 0s - loss: 0.1866 - accuracy: 0.9456
356/375 [===========================>..] - ETA: 0s - loss: 0.1859 - accuracy: 0.9456
362/375 [===========================>..] - ETA: 0s - loss: 0.1854 - accuracy: 0.9457
368/375 [============================>.] - ETA: 0s - loss: 0.1851 - accuracy: 0.9458
373/375 [============================>.] - ETA: 0s - loss: 0.1851 - accuracy: 0.9458
375/375 [==============================] - 3s 9ms/step - loss: 0.1848 - accuracy: 0.9459

375/375 [==============================] - 4s 11ms/step - loss: 0.1848 - accuracy: 0.9459 - val_loss: 0.1861 - val_accuracy: 0.9469 - lr: 0.1000
Epoch 3/20

  1/375 [..............................] - ETA: 3s - loss: 0.1400 - accuracy: 0.9531
  7/375 [..............................] - ETA: 3s - loss: 0.1658 - accuracy: 0.9498
 13/375 [>.............................] - ETA: 3s - loss: 0.1697 - accuracy: 0.9465
 19/375 [>.............................] - ETA: 3s - loss: 0.1627 - accuracy: 0.9482
 25/375 [=>............................] - ETA: 3s - loss: 0.1631 - accuracy: 0.9475
 31/375 [=>............................] - ETA: 3s - loss: 0.1573 - accuracy: 0.9491
 38/375 [==>...........................] - ETA: 2s - loss: 0.1542 - accuracy: 0.9507
 44/375 [==>...........................] - ETA: 2s - loss: 0.1526 - accuracy: 0.9506
 50/375 [===>..........................] - ETA: 2s - loss: 0.1519 - accuracy: 0.9502
 56/375 [===>..........................] - ETA: 2s - loss: 0.1560 - accuracy: 0.9505
 62/375 [===>..........................] - ETA: 2s - loss: 0.1556 - accuracy: 0.9505
 68/375 [====>.........................] - ETA: 2s - loss: 0.1551 - accuracy: 0.9509
 74/375 [====>.........................] - ETA: 2s - loss: 0.1552 - accuracy: 0.9511
 80/375 [=====>........................] - ETA: 2s - loss: 0.1554 - accuracy: 0.9506
 86/375 [=====>........................] - ETA: 2s - loss: 0.1546 - accuracy: 0.9507
 93/375 [======>.......................] - ETA: 2s - loss: 0.1564 - accuracy: 0.9510
 99/375 [======>.......................] - ETA: 2s - loss: 0.1572 - accuracy: 0.9510
105/375 [=======>......................] - ETA: 2s - loss: 0.1548 - accuracy: 0.9516
111/375 [=======>......................] - ETA: 2s - loss: 0.1509 - accuracy: 0.9530
117/375 [========>.....................] - ETA: 2s - loss: 0.1502 - accuracy: 0.9534
123/375 [========>.....................] - ETA: 2s - loss: 0.1500 - accuracy: 0.9538
130/375 [=========>....................] - ETA: 2s - loss: 0.1493 - accuracy: 0.9540
137/375 [=========>....................] - ETA: 2s - loss: 0.1515 - accuracy: 0.9534
143/375 [==========>...................] - ETA: 2s - loss: 0.1519 - accuracy: 0.9534
150/375 [===========>..................] - ETA: 1s - loss: 0.1531 - accuracy: 0.9531
156/375 [===========>..................] - ETA: 1s - loss: 0.1518 - accuracy: 0.9536
161/375 [===========>..................] - ETA: 1s - loss: 0.1540 - accuracy: 0.9528
167/375 [============>.................] - ETA: 1s - loss: 0.1553 - accuracy: 0.9528
173/375 [============>.................] - ETA: 1s - loss: 0.1555 - accuracy: 0.9524
180/375 [=============>................] - ETA: 1s - loss: 0.1567 - accuracy: 0.9522
186/375 [=============>................] - ETA: 1s - loss: 0.1566 - accuracy: 0.9522
193/375 [==============>...............] - ETA: 1s - loss: 0.1573 - accuracy: 0.9519
199/375 [==============>...............] - ETA: 1s - loss: 0.1565 - accuracy: 0.9522
205/375 [===============>..............] - ETA: 1s - loss: 0.1557 - accuracy: 0.9527
209/375 [===============>..............] - ETA: 1s - loss: 0.1557 - accuracy: 0.9528
215/375 [================>.............] - ETA: 1s - loss: 0.1554 - accuracy: 0.9528
221/375 [================>.............] - ETA: 1s - loss: 0.1561 - accuracy: 0.9527
228/375 [=================>............] - ETA: 1s - loss: 0.1561 - accuracy: 0.9526
234/375 [=================>............] - ETA: 1s - loss: 0.1561 - accuracy: 0.9523
240/375 [==================>...........] - ETA: 1s - loss: 0.1553 - accuracy: 0.9524
246/375 [==================>...........] - ETA: 1s - loss: 0.1548 - accuracy: 0.9526
252/375 [===================>..........] - ETA: 1s - loss: 0.1550 - accuracy: 0.9526
258/375 [===================>..........] - ETA: 1s - loss: 0.1549 - accuracy: 0.9525
264/375 [====================>.........] - ETA: 0s - loss: 0.1541 - accuracy: 0.9529
270/375 [====================>.........] - ETA: 0s - loss: 0.1543 - accuracy: 0.9530
276/375 [=====================>........] - ETA: 0s - loss: 0.1540 - accuracy: 0.9531
282/375 [=====================>........] - ETA: 0s - loss: 0.1539 - accuracy: 0.9530
288/375 [======================>.......] - ETA: 0s - loss: 0.1535 - accuracy: 0.9532
294/375 [======================>.......] - ETA: 0s - loss: 0.1532 - accuracy: 0.9533
300/375 [=======================>......] - ETA: 0s - loss: 0.1534 - accuracy: 0.9531
306/375 [=======================>......] - ETA: 0s - loss: 0.1537 - accuracy: 0.9530
312/375 [=======================>......] - ETA: 0s - loss: 0.1532 - accuracy: 0.9532
319/375 [========================>.....] - ETA: 0s - loss: 0.1530 - accuracy: 0.9534
325/375 [=========================>....] - ETA: 0s - loss: 0.1526 - accuracy: 0.9533
332/375 [=========================>....] - ETA: 0s - loss: 0.1520 - accuracy: 0.9536
338/375 [==========================>...] - ETA: 0s - loss: 0.1521 - accuracy: 0.9537
344/375 [==========================>...] - ETA: 0s - loss: 0.1521 - accuracy: 0.9539
350/375 [===========================>..] - ETA: 0s - loss: 0.1519 - accuracy: 0.9538
357/375 [===========================>..] - ETA: 0s - loss: 0.1519 - accuracy: 0.9538
363/375 [============================>.] - ETA: 0s - loss: 0.1520 - accuracy: 0.9538
370/375 [============================>.] - ETA: 0s - loss: 0.1519 - accuracy: 0.9538
375/375 [==============================] - 3s 9ms/step - loss: 0.1516 - accuracy: 0.9539

375/375 [==============================] - 4s 11ms/step - loss: 0.1516 - accuracy: 0.9539 - val_loss: 0.1480 - val_accuracy: 0.9574 - lr: 0.1000
Epoch 4/20

  1/375 [..............................] - ETA: 3s - loss: 0.0867 - accuracy: 0.9844
  7/375 [..............................] - ETA: 3s - loss: 0.0953 - accuracy: 0.9732
 14/375 [>.............................] - ETA: 3s - loss: 0.0941 - accuracy: 0.9743
 20/375 [>.............................] - ETA: 3s - loss: 0.1056 - accuracy: 0.9707
 26/375 [=>............................] - ETA: 3s - loss: 0.1110 - accuracy: 0.9663
 33/375 [=>............................] - ETA: 2s - loss: 0.1170 - accuracy: 0.9645
 40/375 [==>...........................] - ETA: 2s - loss: 0.1164 - accuracy: 0.9645
 47/375 [==>...........................] - ETA: 2s - loss: 0.1192 - accuracy: 0.9634
 52/375 [===>..........................] - ETA: 2s - loss: 0.1183 - accuracy: 0.9627
 58/375 [===>..........................] - ETA: 2s - loss: 0.1195 - accuracy: 0.9627
 65/375 [====>.........................] - ETA: 2s - loss: 0.1182 - accuracy: 0.9632
 71/375 [====>.........................] - ETA: 2s - loss: 0.1204 - accuracy: 0.9631
 77/375 [=====>........................] - ETA: 2s - loss: 0.1203 - accuracy: 0.9625
 83/375 [=====>........................] - ETA: 2s - loss: 0.1180 - accuracy: 0.9630
 90/375 [======>.......................] - ETA: 2s - loss: 0.1158 - accuracy: 0.9641
 96/375 [======>.......................] - ETA: 2s - loss: 0.1150 - accuracy: 0.9641
102/375 [=======>......................] - ETA: 2s - loss: 0.1146 - accuracy: 0.9644
108/375 [=======>......................] - ETA: 2s - loss: 0.1136 - accuracy: 0.9650
114/375 [========>.....................] - ETA: 2s - loss: 0.1130 - accuracy: 0.9655
120/375 [========>.....................] - ETA: 2s - loss: 0.1132 - accuracy: 0.9658
127/375 [=========>....................] - ETA: 2s - loss: 0.1150 - accuracy: 0.9654
133/375 [=========>....................] - ETA: 2s - loss: 0.1134 - accuracy: 0.9659
139/375 [==========>...................] - ETA: 2s - loss: 0.1147 - accuracy: 0.9652
146/375 [==========>...................] - ETA: 1s - loss: 0.1146 - accuracy: 0.9651
153/375 [===========>..................] - ETA: 1s - loss: 0.1140 - accuracy: 0.9651
159/375 [===========>..................] - ETA: 1s - loss: 0.1136 - accuracy: 0.9653
166/375 [============>.................] - ETA: 1s - loss: 0.1137 - accuracy: 0.9655
173/375 [============>.................] - ETA: 1s - loss: 0.1135 - accuracy: 0.9655
180/375 [=============>................] - ETA: 1s - loss: 0.1122 - accuracy: 0.9657
186/375 [=============>................] - ETA: 1s - loss: 0.1125 - accuracy: 0.9657
193/375 [==============>...............] - ETA: 1s - loss: 0.1134 - accuracy: 0.9654
200/375 [===============>..............] - ETA: 1s - loss: 0.1151 - accuracy: 0.9651
207/375 [===============>..............] - ETA: 1s - loss: 0.1158 - accuracy: 0.9647
213/375 [================>.............] - ETA: 1s - loss: 0.1167 - accuracy: 0.9645
220/375 [================>.............] - ETA: 1s - loss: 0.1168 - accuracy: 0.9644
227/375 [=================>............] - ETA: 1s - loss: 0.1177 - accuracy: 0.9642
234/375 [=================>............] - ETA: 1s - loss: 0.1177 - accuracy: 0.9641
240/375 [==================>...........] - ETA: 1s - loss: 0.1188 - accuracy: 0.9638
246/375 [==================>...........] - ETA: 1s - loss: 0.1185 - accuracy: 0.9636
252/375 [===================>..........] - ETA: 1s - loss: 0.1186 - accuracy: 0.9636
258/375 [===================>..........] - ETA: 0s - loss: 0.1183 - accuracy: 0.9636
264/375 [====================>.........] - ETA: 0s - loss: 0.1186 - accuracy: 0.9635
269/375 [====================>.........] - ETA: 0s - loss: 0.1186 - accuracy: 0.9634
275/375 [=====================>........] - ETA: 0s - loss: 0.1192 - accuracy: 0.9632
281/375 [=====================>........] - ETA: 0s - loss: 0.1199 - accuracy: 0.9630
287/375 [=====================>........] - ETA: 0s - loss: 0.1206 - accuracy: 0.9629
293/375 [======================>.......] - ETA: 0s - loss: 0.1208 - accuracy: 0.9629
299/375 [======================>.......] - ETA: 0s - loss: 0.1208 - accuracy: 0.9629
305/375 [=======================>......] - ETA: 0s - loss: 0.1211 - accuracy: 0.9630
311/375 [=======================>......] - ETA: 0s - loss: 0.1214 - accuracy: 0.9629
317/375 [========================>.....] - ETA: 0s - loss: 0.1217 - accuracy: 0.9629
323/375 [========================>.....] - ETA: 0s - loss: 0.1228 - accuracy: 0.9627
330/375 [=========================>....] - ETA: 0s - loss: 0.1235 - accuracy: 0.9623
336/375 [=========================>....] - ETA: 0s - loss: 0.1237 - accuracy: 0.9623
342/375 [==========================>...] - ETA: 0s - loss: 0.1239 - accuracy: 0.9622
347/375 [==========================>...] - ETA: 0s - loss: 0.1241 - accuracy: 0.9621
353/375 [===========================>..] - ETA: 0s - loss: 0.1232 - accuracy: 0.9625
358/375 [===========================>..] - ETA: 0s - loss: 0.1234 - accuracy: 0.9624
364/375 [============================>.] - ETA: 0s - loss: 0.1236 - accuracy: 0.9623
370/375 [============================>.] - ETA: 0s - loss: 0.1235 - accuracy: 0.9625
375/375 [==============================] - 3s 9ms/step - loss: 0.1230 - accuracy: 0.9627

375/375 [==============================] - 4s 11ms/step - loss: 0.1230 - accuracy: 0.9627 - val_loss: 0.1465 - val_accuracy: 0.9580 - lr: 0.1000
Epoch 5/20

  1/375 [..............................] - ETA: 3s - loss: 0.0228 - accuracy: 1.0000
  7/375 [..............................] - ETA: 3s - loss: 0.0856 - accuracy: 0.9699
 14/375 [>.............................] - ETA: 3s - loss: 0.0996 - accuracy: 0.9676
 21/375 [>.............................] - ETA: 2s - loss: 0.0936 - accuracy: 0.9688
 28/375 [=>............................] - ETA: 2s - loss: 0.0918 - accuracy: 0.9701
 35/375 [=>............................] - ETA: 2s - loss: 0.0970 - accuracy: 0.9692
 41/375 [==>...........................] - ETA: 2s - loss: 0.0970 - accuracy: 0.9693
 47/375 [==>...........................] - ETA: 2s - loss: 0.1000 - accuracy: 0.9683
 53/375 [===>..........................] - ETA: 2s - loss: 0.1002 - accuracy: 0.9680
 59/375 [===>..........................] - ETA: 2s - loss: 0.1030 - accuracy: 0.9669
 65/375 [====>.........................] - ETA: 2s - loss: 0.1060 - accuracy: 0.9661
 71/375 [====>.........................] - ETA: 2s - loss: 0.1076 - accuracy: 0.9660
 77/375 [=====>........................] - ETA: 2s - loss: 0.1092 - accuracy: 0.9662
 83/375 [=====>........................] - ETA: 2s - loss: 0.1090 - accuracy: 0.9667
 89/375 [======>.......................] - ETA: 2s - loss: 0.1089 - accuracy: 0.9671
 95/375 [======>.......................] - ETA: 2s - loss: 0.1098 - accuracy: 0.9666
102/375 [=======>......................] - ETA: 2s - loss: 0.1088 - accuracy: 0.9667
108/375 [=======>......................] - ETA: 2s - loss: 0.1075 - accuracy: 0.9672
114/375 [========>.....................] - ETA: 2s - loss: 0.1070 - accuracy: 0.9672
120/375 [========>.....................] - ETA: 2s - loss: 0.1054 - accuracy: 0.9676
126/375 [=========>....................] - ETA: 2s - loss: 0.1054 - accuracy: 0.9678
133/375 [=========>....................] - ETA: 2s - loss: 0.1033 - accuracy: 0.9685
139/375 [==========>...................] - ETA: 2s - loss: 0.1030 - accuracy: 0.9685
146/375 [==========>...................] - ETA: 1s - loss: 0.1024 - accuracy: 0.9686
152/375 [===========>..................] - ETA: 1s - loss: 0.1022 - accuracy: 0.9687
158/375 [===========>..................] - ETA: 1s - loss: 0.1020 - accuracy: 0.9685
164/375 [============>.................] - ETA: 1s - loss: 0.1035 - accuracy: 0.9686
171/375 [============>.................] - ETA: 1s - loss: 0.1034 - accuracy: 0.9687
178/375 [=============>................] - ETA: 1s - loss: 0.1045 - accuracy: 0.9683
184/375 [=============>................] - ETA: 1s - loss: 0.1044 - accuracy: 0.9682
189/375 [==============>...............] - ETA: 1s - loss: 0.1051 - accuracy: 0.9680
194/375 [==============>...............] - ETA: 1s - loss: 0.1053 - accuracy: 0.9680
200/375 [===============>..............] - ETA: 1s - loss: 0.1055 - accuracy: 0.9680
206/375 [===============>..............] - ETA: 1s - loss: 0.1067 - accuracy: 0.9675
212/375 [===============>..............] - ETA: 1s - loss: 0.1075 - accuracy: 0.9674
219/375 [================>.............] - ETA: 1s - loss: 0.1074 - accuracy: 0.9673
225/375 [=================>............] - ETA: 1s - loss: 0.1071 - accuracy: 0.9674
231/375 [=================>............] - ETA: 1s - loss: 0.1083 - accuracy: 0.9672
237/375 [=================>............] - ETA: 1s - loss: 0.1077 - accuracy: 0.9674
243/375 [==================>...........] - ETA: 1s - loss: 0.1083 - accuracy: 0.9672
249/375 [==================>...........] - ETA: 1s - loss: 0.1080 - accuracy: 0.9674
255/375 [===================>..........] - ETA: 1s - loss: 0.1071 - accuracy: 0.9675
261/375 [===================>..........] - ETA: 0s - loss: 0.1070 - accuracy: 0.9676
268/375 [====================>.........] - ETA: 0s - loss: 0.1073 - accuracy: 0.9675
274/375 [====================>.........] - ETA: 0s - loss: 0.1073 - accuracy: 0.9675
280/375 [=====================>........] - ETA: 0s - loss: 0.1072 - accuracy: 0.9676
287/375 [=====================>........] - ETA: 0s - loss: 0.1072 - accuracy: 0.9677
293/375 [======================>.......] - ETA: 0s - loss: 0.1077 - accuracy: 0.9676
300/375 [=======================>......] - ETA: 0s - loss: 0.1073 - accuracy: 0.9677
306/375 [=======================>......] - ETA: 0s - loss: 0.1067 - accuracy: 0.9679
312/375 [=======================>......] - ETA: 0s - loss: 0.1070 - accuracy: 0.9678
318/375 [========================>.....] - ETA: 0s - loss: 0.1069 - accuracy: 0.9678
325/375 [=========================>....] - ETA: 0s - loss: 0.1073 - accuracy: 0.9677
332/375 [=========================>....] - ETA: 0s - loss: 0.1072 - accuracy: 0.9678
338/375 [==========================>...] - ETA: 0s - loss: 0.1075 - accuracy: 0.9679
344/375 [==========================>...] - ETA: 0s - loss: 0.1077 - accuracy: 0.9678
350/375 [===========================>..] - ETA: 0s - loss: 0.1079 - accuracy: 0.9677
356/375 [===========================>..] - ETA: 0s - loss: 0.1080 - accuracy: 0.9675
363/375 [============================>.] - ETA: 0s - loss: 0.1093 - accuracy: 0.9672
370/375 [============================>.] - ETA: 0s - loss: 0.1095 - accuracy: 0.9672
375/375 [==============================] - 3s 9ms/step - loss: 0.1098 - accuracy: 0.9670

375/375 [==============================] - 4s 11ms/step - loss: 0.1098 - accuracy: 0.9670 - val_loss: 0.1412 - val_accuracy: 0.9592 - lr: 0.1000
Epoch 6/20

  1/375 [..............................] - ETA: 3s - loss: 0.1542 - accuracy: 0.9531
  8/375 [..............................] - ETA: 3s - loss: 0.0903 - accuracy: 0.9678
 14/375 [>.............................] - ETA: 3s - loss: 0.1039 - accuracy: 0.9648
 21/375 [>.............................] - ETA: 2s - loss: 0.1013 - accuracy: 0.9661
 28/375 [=>............................] - ETA: 2s - loss: 0.0964 - accuracy: 0.9688
 35/375 [=>............................] - ETA: 2s - loss: 0.0912 - accuracy: 0.9701
 42/375 [==>...........................] - ETA: 2s - loss: 0.0862 - accuracy: 0.9721
 48/375 [==>...........................] - ETA: 2s - loss: 0.0843 - accuracy: 0.9730
 54/375 [===>..........................] - ETA: 2s - loss: 0.0845 - accuracy: 0.9732
 60/375 [===>..........................] - ETA: 2s - loss: 0.0851 - accuracy: 0.9728
 67/375 [====>.........................] - ETA: 2s - loss: 0.0849 - accuracy: 0.9729
 73/375 [====>.........................] - ETA: 2s - loss: 0.0837 - accuracy: 0.9738
 79/375 [=====>........................] - ETA: 2s - loss: 0.0841 - accuracy: 0.9742
 85/375 [=====>........................] - ETA: 2s - loss: 0.0831 - accuracy: 0.9747
 91/375 [======>.......................] - ETA: 2s - loss: 0.0840 - accuracy: 0.9745
 98/375 [======>.......................] - ETA: 2s - loss: 0.0833 - accuracy: 0.9746
104/375 [=======>......................] - ETA: 2s - loss: 0.0854 - accuracy: 0.9743
110/375 [=======>......................] - ETA: 2s - loss: 0.0878 - accuracy: 0.9738
116/375 [========>.....................] - ETA: 2s - loss: 0.0892 - accuracy: 0.9733
122/375 [========>.....................] - ETA: 2s - loss: 0.0907 - accuracy: 0.9727
128/375 [=========>....................] - ETA: 2s - loss: 0.0899 - accuracy: 0.9728
134/375 [=========>....................] - ETA: 2s - loss: 0.0912 - accuracy: 0.9725
140/375 [==========>...................] - ETA: 1s - loss: 0.0911 - accuracy: 0.9722
146/375 [==========>...................] - ETA: 1s - loss: 0.0902 - accuracy: 0.9725
153/375 [===========>..................] - ETA: 1s - loss: 0.0907 - accuracy: 0.9723
159/375 [===========>..................] - ETA: 1s - loss: 0.0913 - accuracy: 0.9721
166/375 [============>.................] - ETA: 1s - loss: 0.0913 - accuracy: 0.9721
173/375 [============>.................] - ETA: 1s - loss: 0.0922 - accuracy: 0.9717
179/375 [=============>................] - ETA: 1s - loss: 0.0916 - accuracy: 0.9721
186/375 [=============>................] - ETA: 1s - loss: 0.0911 - accuracy: 0.9724
193/375 [==============>...............] - ETA: 1s - loss: 0.0910 - accuracy: 0.9723
199/375 [==============>...............] - ETA: 1s - loss: 0.0917 - accuracy: 0.9722
205/375 [===============>..............] - ETA: 1s - loss: 0.0916 - accuracy: 0.9724
212/375 [===============>..............] - ETA: 1s - loss: 0.0925 - accuracy: 0.9719
219/375 [================>.............] - ETA: 1s - loss: 0.0929 - accuracy: 0.9717
225/375 [=================>............] - ETA: 1s - loss: 0.0939 - accuracy: 0.9712
232/375 [=================>............] - ETA: 1s - loss: 0.0936 - accuracy: 0.9714
239/375 [==================>...........] - ETA: 1s - loss: 0.0937 - accuracy: 0.9713
245/375 [==================>...........] - ETA: 1s - loss: 0.0945 - accuracy: 0.9711
252/375 [===================>..........] - ETA: 1s - loss: 0.0944 - accuracy: 0.9710
258/375 [===================>..........] - ETA: 0s - loss: 0.0955 - accuracy: 0.9707
265/375 [====================>.........] - ETA: 0s - loss: 0.0959 - accuracy: 0.9706
271/375 [====================>.........] - ETA: 0s - loss: 0.0964 - accuracy: 0.9705
277/375 [=====================>........] - ETA: 0s - loss: 0.0971 - accuracy: 0.9703
284/375 [=====================>........] - ETA: 0s - loss: 0.0965 - accuracy: 0.9704
290/375 [======================>.......] - ETA: 0s - loss: 0.0965 - accuracy: 0.9705
296/375 [======================>.......] - ETA: 0s - loss: 0.0969 - accuracy: 0.9704
302/375 [=======================>......] - ETA: 0s - loss: 0.0966 - accuracy: 0.9703
308/375 [=======================>......] - ETA: 0s - loss: 0.0975 - accuracy: 0.9701
315/375 [========================>.....] - ETA: 0s - loss: 0.0980 - accuracy: 0.9700
321/375 [========================>.....] - ETA: 0s - loss: 0.0981 - accuracy: 0.9700
323/375 [========================>.....] - ETA: 0s - loss: 0.0982 - accuracy: 0.9700
329/375 [=========================>....] - ETA: 0s - loss: 0.0985 - accuracy: 0.9698
335/375 [=========================>....] - ETA: 0s - loss: 0.0985 - accuracy: 0.9698
341/375 [==========================>...] - ETA: 0s - loss: 0.0986 - accuracy: 0.9696
347/375 [==========================>...] - ETA: 0s - loss: 0.0987 - accuracy: 0.9697
353/375 [===========================>..] - ETA: 0s - loss: 0.0992 - accuracy: 0.9696
358/375 [===========================>..] - ETA: 0s - loss: 0.0998 - accuracy: 0.9693
364/375 [============================>.] - ETA: 0s - loss: 0.0993 - accuracy: 0.9694
370/375 [============================>.] - ETA: 0s - loss: 0.0996 - accuracy: 0.9693
375/375 [==============================] - 3s 9ms/step - loss: 0.0998 - accuracy: 0.9691

375/375 [==============================] - 4s 11ms/step - loss: 0.0998 - accuracy: 0.9691 - val_loss: 0.1363 - val_accuracy: 0.9612 - lr: 0.1000
Epoch 7/20

  1/375 [..............................] - ETA: 2s - loss: 0.0257 - accuracy: 1.0000
  7/375 [..............................] - ETA: 3s - loss: 0.0649 - accuracy: 0.9855
 13/375 [>.............................] - ETA: 3s - loss: 0.0561 - accuracy: 0.9856
 16/375 [>.............................] - ETA: 3s - loss: 0.0560 - accuracy: 0.9858
 23/375 [>.............................] - ETA: 3s - loss: 0.0625 - accuracy: 0.9820
 30/375 [=>............................] - ETA: 3s - loss: 0.0640 - accuracy: 0.9813
 37/375 [=>............................] - ETA: 3s - loss: 0.0724 - accuracy: 0.9789
 43/375 [==>...........................] - ETA: 3s - loss: 0.0713 - accuracy: 0.9795
 49/375 [==>...........................] - ETA: 3s - loss: 0.0685 - accuracy: 0.9796
 56/375 [===>..........................] - ETA: 2s - loss: 0.0691 - accuracy: 0.9791
 62/375 [===>..........................] - ETA: 2s - loss: 0.0713 - accuracy: 0.9783
 68/375 [====>.........................] - ETA: 2s - loss: 0.0703 - accuracy: 0.9783
 74/375 [====>.........................] - ETA: 2s - loss: 0.0723 - accuracy: 0.9775
 80/375 [=====>........................] - ETA: 2s - loss: 0.0738 - accuracy: 0.9771
 86/375 [=====>........................] - ETA: 2s - loss: 0.0758 - accuracy: 0.9767
 92/375 [======>.......................] - ETA: 2s - loss: 0.0756 - accuracy: 0.9766
 98/375 [======>.......................] - ETA: 2s - loss: 0.0745 - accuracy: 0.9767
104/375 [=======>......................] - ETA: 2s - loss: 0.0743 - accuracy: 0.9767
110/375 [=======>......................] - ETA: 2s - loss: 0.0737 - accuracy: 0.9768
116/375 [========>.....................] - ETA: 2s - loss: 0.0761 - accuracy: 0.9763
123/375 [========>.....................] - ETA: 2s - loss: 0.0773 - accuracy: 0.9761
129/375 [=========>....................] - ETA: 2s - loss: 0.0782 - accuracy: 0.9755
135/375 [=========>....................] - ETA: 2s - loss: 0.0792 - accuracy: 0.9751
141/375 [==========>...................] - ETA: 2s - loss: 0.0805 - accuracy: 0.9746
147/375 [==========>...................] - ETA: 2s - loss: 0.0799 - accuracy: 0.9750
153/375 [===========>..................] - ETA: 1s - loss: 0.0798 - accuracy: 0.9749
159/375 [===========>..................] - ETA: 1s - loss: 0.0808 - accuracy: 0.9747
165/375 [============>.................] - ETA: 1s - loss: 0.0813 - accuracy: 0.9744
171/375 [============>.................] - ETA: 1s - loss: 0.0815 - accuracy: 0.9746
178/375 [=============>................] - ETA: 1s - loss: 0.0830 - accuracy: 0.9745
184/375 [=============>................] - ETA: 1s - loss: 0.0840 - accuracy: 0.9743
190/375 [==============>...............] - ETA: 1s - loss: 0.0846 - accuracy: 0.9740
196/375 [==============>...............] - ETA: 1s - loss: 0.0866 - accuracy: 0.9733
202/375 [===============>..............] - ETA: 1s - loss: 0.0862 - accuracy: 0.9735
207/375 [===============>..............] - ETA: 1s - loss: 0.0864 - accuracy: 0.9732
213/375 [================>.............] - ETA: 1s - loss: 0.0861 - accuracy: 0.9731
219/375 [================>.............] - ETA: 1s - loss: 0.0870 - accuracy: 0.9729
225/375 [=================>............] - ETA: 1s - loss: 0.0887 - accuracy: 0.9724
231/375 [=================>............] - ETA: 1s - loss: 0.0897 - accuracy: 0.9722
237/375 [=================>............] - ETA: 1s - loss: 0.0904 - accuracy: 0.9720
243/375 [==================>...........] - ETA: 1s - loss: 0.0905 - accuracy: 0.9718
249/375 [==================>...........] - ETA: 1s - loss: 0.0908 - accuracy: 0.9718
255/375 [===================>..........] - ETA: 1s - loss: 0.0910 - accuracy: 0.9717
262/375 [===================>..........] - ETA: 1s - loss: 0.0913 - accuracy: 0.9716
268/375 [====================>.........] - ETA: 0s - loss: 0.0911 - accuracy: 0.9716
274/375 [====================>.........] - ETA: 0s - loss: 0.0912 - accuracy: 0.9717
280/375 [=====================>........] - ETA: 0s - loss: 0.0908 - accuracy: 0.9718
286/375 [=====================>........] - ETA: 0s - loss: 0.0906 - accuracy: 0.9720
293/375 [======================>.......] - ETA: 0s - loss: 0.0899 - accuracy: 0.9721
299/375 [======================>.......] - ETA: 0s - loss: 0.0898 - accuracy: 0.9720
306/375 [=======================>......] - ETA: 0s - loss: 0.0898 - accuracy: 0.9720
312/375 [=======================>......] - ETA: 0s - loss: 0.0905 - accuracy: 0.9720
318/375 [========================>.....] - ETA: 0s - loss: 0.0905 - accuracy: 0.9719
324/375 [========================>.....] - ETA: 0s - loss: 0.0906 - accuracy: 0.9720
330/375 [=========================>....] - ETA: 0s - loss: 0.0903 - accuracy: 0.9719
336/375 [=========================>....] - ETA: 0s - loss: 0.0905 - accuracy: 0.9718
342/375 [==========================>...] - ETA: 0s - loss: 0.0901 - accuracy: 0.9719
349/375 [==========================>...] - ETA: 0s - loss: 0.0909 - accuracy: 0.9717
355/375 [===========================>..] - ETA: 0s - loss: 0.0911 - accuracy: 0.9716
361/375 [===========================>..] - ETA: 0s - loss: 0.0910 - accuracy: 0.9718
368/375 [============================>.] - ETA: 0s - loss: 0.0908 - accuracy: 0.9717
374/375 [============================>.] - ETA: 0s - loss: 0.0908 - accuracy: 0.9717
375/375 [==============================] - 3s 9ms/step - loss: 0.0908 - accuracy: 0.9716

375/375 [==============================] - 4s 11ms/step - loss: 0.0908 - accuracy: 0.9716 - val_loss: 0.1305 - val_accuracy: 0.9619 - lr: 0.1000
Epoch 8/20

  1/375 [..............................] - ETA: 3s - loss: 0.0387 - accuracy: 0.9922
  8/375 [..............................] - ETA: 2s - loss: 0.0705 - accuracy: 0.9834
 14/375 [>.............................] - ETA: 3s - loss: 0.0782 - accuracy: 0.9799
 20/375 [>.............................] - ETA: 3s - loss: 0.0807 - accuracy: 0.9762
 26/375 [=>............................] - ETA: 2s - loss: 0.0789 - accuracy: 0.9751
 32/375 [=>............................] - ETA: 2s - loss: 0.0769 - accuracy: 0.9761
 38/375 [==>...........................] - ETA: 2s - loss: 0.0866 - accuracy: 0.9727
 44/375 [==>...........................] - ETA: 2s - loss: 0.0867 - accuracy: 0.9730
 50/375 [===>..........................] - ETA: 2s - loss: 0.0877 - accuracy: 0.9725
 56/375 [===>..........................] - ETA: 2s - loss: 0.0864 - accuracy: 0.9731
 63/375 [====>.........................] - ETA: 2s - loss: 0.0844 - accuracy: 0.9732
 69/375 [====>.........................] - ETA: 2s - loss: 0.0843 - accuracy: 0.9733
 76/375 [=====>........................] - ETA: 2s - loss: 0.0819 - accuracy: 0.9743
 82/375 [=====>........................] - ETA: 2s - loss: 0.0811 - accuracy: 0.9748
 89/375 [======>.......................] - ETA: 2s - loss: 0.0818 - accuracy: 0.9745
 96/375 [======>.......................] - ETA: 2s - loss: 0.0829 - accuracy: 0.9741
102/375 [=======>......................] - ETA: 2s - loss: 0.0832 - accuracy: 0.9737
108/375 [=======>......................] - ETA: 2s - loss: 0.0844 - accuracy: 0.9729
114/375 [========>.....................] - ETA: 2s - loss: 0.0834 - accuracy: 0.9731
120/375 [========>.....................] - ETA: 2s - loss: 0.0844 - accuracy: 0.9729
127/375 [=========>....................] - ETA: 2s - loss: 0.0844 - accuracy: 0.9728
133/375 [=========>....................] - ETA: 2s - loss: 0.0851 - accuracy: 0.9727
139/375 [==========>...................] - ETA: 2s - loss: 0.0851 - accuracy: 0.9729
145/375 [==========>...................] - ETA: 1s - loss: 0.0850 - accuracy: 0.9729
151/375 [===========>..................] - ETA: 1s - loss: 0.0842 - accuracy: 0.9730
157/375 [===========>..................] - ETA: 1s - loss: 0.0847 - accuracy: 0.9726
163/375 [============>.................] - ETA: 1s - loss: 0.0852 - accuracy: 0.9725
169/375 [============>.................] - ETA: 1s - loss: 0.0849 - accuracy: 0.9726
176/375 [=============>................] - ETA: 1s - loss: 0.0851 - accuracy: 0.9726
183/375 [=============>................] - ETA: 1s - loss: 0.0852 - accuracy: 0.9727
189/375 [==============>...............] - ETA: 1s - loss: 0.0848 - accuracy: 0.9729
196/375 [==============>...............] - ETA: 1s - loss: 0.0852 - accuracy: 0.9727
202/375 [===============>..............] - ETA: 1s - loss: 0.0850 - accuracy: 0.9729
209/375 [===============>..............] - ETA: 1s - loss: 0.0858 - accuracy: 0.9726
216/375 [================>.............] - ETA: 1s - loss: 0.0852 - accuracy: 0.9728
223/375 [================>.............] - ETA: 1s - loss: 0.0856 - accuracy: 0.9726
229/375 [=================>............] - ETA: 1s - loss: 0.0850 - accuracy: 0.9728
235/375 [=================>............] - ETA: 1s - loss: 0.0846 - accuracy: 0.9730
242/375 [==================>...........] - ETA: 1s - loss: 0.0846 - accuracy: 0.9729
248/375 [==================>...........] - ETA: 1s - loss: 0.0857 - accuracy: 0.9726
254/375 [===================>..........] - ETA: 1s - loss: 0.0856 - accuracy: 0.9727
260/375 [===================>..........] - ETA: 0s - loss: 0.0856 - accuracy: 0.9727
266/375 [====================>.........] - ETA: 0s - loss: 0.0853 - accuracy: 0.9729
272/375 [====================>.........] - ETA: 0s - loss: 0.0852 - accuracy: 0.9731
278/375 [=====================>........] - ETA: 0s - loss: 0.0847 - accuracy: 0.9733
284/375 [=====================>........] - ETA: 0s - loss: 0.0845 - accuracy: 0.9735
291/375 [======================>.......] - ETA: 0s - loss: 0.0850 - accuracy: 0.9734
297/375 [======================>.......] - ETA: 0s - loss: 0.0856 - accuracy: 0.9733
303/375 [=======================>......] - ETA: 0s - loss: 0.0860 - accuracy: 0.9732
309/375 [=======================>......] - ETA: 0s - loss: 0.0859 - accuracy: 0.9732
315/375 [========================>.....] - ETA: 0s - loss: 0.0859 - accuracy: 0.9733
321/375 [========================>.....] - ETA: 0s - loss: 0.0860 - accuracy: 0.9732
327/375 [=========================>....] - ETA: 0s - loss: 0.0860 - accuracy: 0.9731
333/375 [=========================>....] - ETA: 0s - loss: 0.0855 - accuracy: 0.9733
340/375 [==========================>...] - ETA: 0s - loss: 0.0852 - accuracy: 0.9734
346/375 [==========================>...] - ETA: 0s - loss: 0.0857 - accuracy: 0.9734
352/375 [===========================>..] - ETA: 0s - loss: 0.0854 - accuracy: 0.9736
359/375 [===========================>..] - ETA: 0s - loss: 0.0850 - accuracy: 0.9737
365/375 [============================>.] - ETA: 0s - loss: 0.0849 - accuracy: 0.9738
371/375 [============================>.] - ETA: 0s - loss: 0.0848 - accuracy: 0.9739
375/375 [==============================] - 3s 9ms/step - loss: 0.0848 - accuracy: 0.9739

375/375 [==============================] - 4s 11ms/step - loss: 0.0848 - accuracy: 0.9739 - val_loss: 0.1262 - val_accuracy: 0.9648 - lr: 0.1000
Epoch 9/20

  1/375 [..............................] - ETA: 4s - loss: 0.0324 - accuracy: 0.9844
  7/375 [..............................] - ETA: 3s - loss: 0.0691 - accuracy: 0.9721
 14/375 [>.............................] - ETA: 3s - loss: 0.0682 - accuracy: 0.9760
 21/375 [>.............................] - ETA: 2s - loss: 0.0667 - accuracy: 0.9766
 27/375 [=>............................] - ETA: 2s - loss: 0.0674 - accuracy: 0.9769
 32/375 [=>............................] - ETA: 3s - loss: 0.0637 - accuracy: 0.9773
 36/375 [=>............................] - ETA: 3s - loss: 0.0644 - accuracy: 0.9772
 42/375 [==>...........................] - ETA: 3s - loss: 0.0654 - accuracy: 0.9758
 48/375 [==>...........................] - ETA: 2s - loss: 0.0650 - accuracy: 0.9766
 54/375 [===>..........................] - ETA: 2s - loss: 0.0640 - accuracy: 0.9774
 59/375 [===>..........................] - ETA: 2s - loss: 0.0636 - accuracy: 0.9774
 65/375 [====>.........................] - ETA: 2s - loss: 0.0651 - accuracy: 0.9773
 71/375 [====>.........................] - ETA: 2s - loss: 0.0663 - accuracy: 0.9772
 75/375 [=====>........................] - ETA: 2s - loss: 0.0657 - accuracy: 0.9775
 81/375 [=====>........................] - ETA: 2s - loss: 0.0674 - accuracy: 0.9769
 87/375 [=====>........................] - ETA: 2s - loss: 0.0686 - accuracy: 0.9769
 93/375 [======>.......................] - ETA: 2s - loss: 0.0696 - accuracy: 0.9764
 99/375 [======>.......................] - ETA: 2s - loss: 0.0713 - accuracy: 0.9755
105/375 [=======>......................] - ETA: 2s - loss: 0.0706 - accuracy: 0.9757
112/375 [=======>......................] - ETA: 2s - loss: 0.0738 - accuracy: 0.9751
118/375 [========>.....................] - ETA: 2s - loss: 0.0749 - accuracy: 0.9748
124/375 [========>.....................] - ETA: 2s - loss: 0.0757 - accuracy: 0.9747
130/375 [=========>....................] - ETA: 2s - loss: 0.0752 - accuracy: 0.9748
136/375 [=========>....................] - ETA: 2s - loss: 0.0761 - accuracy: 0.9747
142/375 [==========>...................] - ETA: 2s - loss: 0.0776 - accuracy: 0.9744
149/375 [==========>...................] - ETA: 2s - loss: 0.0778 - accuracy: 0.9744
155/375 [===========>..................] - ETA: 1s - loss: 0.0776 - accuracy: 0.9746
161/375 [===========>..................] - ETA: 1s - loss: 0.0792 - accuracy: 0.9742
167/375 [============>.................] - ETA: 1s - loss: 0.0798 - accuracy: 0.9739
173/375 [============>.................] - ETA: 1s - loss: 0.0803 - accuracy: 0.9739
179/375 [=============>................] - ETA: 1s - loss: 0.0801 - accuracy: 0.9740
186/375 [=============>................] - ETA: 1s - loss: 0.0800 - accuracy: 0.9741
192/375 [==============>...............] - ETA: 1s - loss: 0.0800 - accuracy: 0.9741
199/375 [==============>...............] - ETA: 1s - loss: 0.0798 - accuracy: 0.9741
205/375 [===============>..............] - ETA: 1s - loss: 0.0788 - accuracy: 0.9745
211/375 [===============>..............] - ETA: 1s - loss: 0.0796 - accuracy: 0.9741
217/375 [================>.............] - ETA: 1s - loss: 0.0796 - accuracy: 0.9742
224/375 [================>.............] - ETA: 1s - loss: 0.0797 - accuracy: 0.9742
230/375 [=================>............] - ETA: 1s - loss: 0.0796 - accuracy: 0.9743
236/375 [=================>............] - ETA: 1s - loss: 0.0797 - accuracy: 0.9742
242/375 [==================>...........] - ETA: 1s - loss: 0.0812 - accuracy: 0.9739
248/375 [==================>...........] - ETA: 1s - loss: 0.0817 - accuracy: 0.9737
254/375 [===================>..........] - ETA: 1s - loss: 0.0810 - accuracy: 0.9739
260/375 [===================>..........] - ETA: 1s - loss: 0.0809 - accuracy: 0.9740
267/375 [====================>.........] - ETA: 0s - loss: 0.0807 - accuracy: 0.9740
274/375 [====================>.........] - ETA: 0s - loss: 0.0814 - accuracy: 0.9739
280/375 [=====================>........] - ETA: 0s - loss: 0.0820 - accuracy: 0.9739
286/375 [=====================>........] - ETA: 0s - loss: 0.0822 - accuracy: 0.9737
292/375 [======================>.......] - ETA: 0s - loss: 0.0821 - accuracy: 0.9736
298/375 [======================>.......] - ETA: 0s - loss: 0.0820 - accuracy: 0.9735
304/375 [=======================>......] - ETA: 0s - loss: 0.0825 - accuracy: 0.9734
310/375 [=======================>......] - ETA: 0s - loss: 0.0821 - accuracy: 0.9736
316/375 [========================>.....] - ETA: 0s - loss: 0.0815 - accuracy: 0.9738
322/375 [========================>.....] - ETA: 0s - loss: 0.0812 - accuracy: 0.9738
329/375 [=========================>....] - ETA: 0s - loss: 0.0809 - accuracy: 0.9740
336/375 [=========================>....] - ETA: 0s - loss: 0.0820 - accuracy: 0.9737
342/375 [==========================>...] - ETA: 0s - loss: 0.0816 - accuracy: 0.9738
348/375 [==========================>...] - ETA: 0s - loss: 0.0818 - accuracy: 0.9737
355/375 [===========================>..] - ETA: 0s - loss: 0.0823 - accuracy: 0.9737
362/375 [===========================>..] - ETA: 0s - loss: 0.0820 - accuracy: 0.9738
368/375 [============================>.] - ETA: 0s - loss: 0.0821 - accuracy: 0.9738
375/375 [==============================] - 3s 9ms/step - loss: 0.0817 - accuracy: 0.9740

375/375 [==============================] - 4s 11ms/step - loss: 0.0817 - accuracy: 0.9740 - val_loss: 0.1198 - val_accuracy: 0.9677 - lr: 0.1000
Epoch 10/20

  1/375 [..............................] - ETA: 3s - loss: 0.0967 - accuracy: 0.9688
  7/375 [..............................] - ETA: 3s - loss: 0.0622 - accuracy: 0.9821
 13/375 [>.............................] - ETA: 3s - loss: 0.0541 - accuracy: 0.9838
 19/375 [>.............................] - ETA: 3s - loss: 0.0541 - accuracy: 0.9815
 26/375 [=>............................] - ETA: 2s - loss: 0.0718 - accuracy: 0.9757
 33/375 [=>............................] - ETA: 2s - loss: 0.0751 - accuracy: 0.9754
 39/375 [==>...........................] - ETA: 2s - loss: 0.0755 - accuracy: 0.9756
 45/375 [==>...........................] - ETA: 2s - loss: 0.0790 - accuracy: 0.9747
 51/375 [===>..........................] - ETA: 2s - loss: 0.0798 - accuracy: 0.9746
 57/375 [===>..........................] - ETA: 2s - loss: 0.0784 - accuracy: 0.9749
 63/375 [====>.........................] - ETA: 2s - loss: 0.0758 - accuracy: 0.9761
 69/375 [====>.........................] - ETA: 2s - loss: 0.0742 - accuracy: 0.9767
 75/375 [=====>........................] - ETA: 2s - loss: 0.0727 - accuracy: 0.9770
 82/375 [=====>........................] - ETA: 2s - loss: 0.0720 - accuracy: 0.9770
 88/375 [======>.......................] - ETA: 2s - loss: 0.0725 - accuracy: 0.9769
 94/375 [======>.......................] - ETA: 2s - loss: 0.0723 - accuracy: 0.9771
101/375 [=======>......................] - ETA: 2s - loss: 0.0722 - accuracy: 0.9775
107/375 [=======>......................] - ETA: 2s - loss: 0.0731 - accuracy: 0.9774
113/375 [========>.....................] - ETA: 2s - loss: 0.0729 - accuracy: 0.9775
119/375 [========>.....................] - ETA: 2s - loss: 0.0733 - accuracy: 0.9774
126/375 [=========>....................] - ETA: 2s - loss: 0.0726 - accuracy: 0.9775
132/375 [=========>....................] - ETA: 2s - loss: 0.0746 - accuracy: 0.9772
139/375 [==========>...................] - ETA: 2s - loss: 0.0751 - accuracy: 0.9770
144/375 [==========>...................] - ETA: 1s - loss: 0.0750 - accuracy: 0.9770
151/375 [===========>..................] - ETA: 1s - loss: 0.0735 - accuracy: 0.9774
157/375 [===========>..................] - ETA: 1s - loss: 0.0732 - accuracy: 0.9776
163/375 [============>.................] - ETA: 1s - loss: 0.0731 - accuracy: 0.9777
169/375 [============>.................] - ETA: 1s - loss: 0.0728 - accuracy: 0.9777
175/375 [=============>................] - ETA: 1s - loss: 0.0726 - accuracy: 0.9775
181/375 [=============>................] - ETA: 1s - loss: 0.0718 - accuracy: 0.9778
187/375 [=============>................] - ETA: 1s - loss: 0.0708 - accuracy: 0.9780
194/375 [==============>...............] - ETA: 1s - loss: 0.0712 - accuracy: 0.9778
200/375 [===============>..............] - ETA: 1s - loss: 0.0713 - accuracy: 0.9776
207/375 [===============>..............] - ETA: 1s - loss: 0.0728 - accuracy: 0.9772
214/375 [================>.............] - ETA: 1s - loss: 0.0730 - accuracy: 0.9769
221/375 [================>.............] - ETA: 1s - loss: 0.0736 - accuracy: 0.9768
227/375 [=================>............] - ETA: 1s - loss: 0.0740 - accuracy: 0.9768
233/375 [=================>............] - ETA: 1s - loss: 0.0735 - accuracy: 0.9769
239/375 [==================>...........] - ETA: 1s - loss: 0.0730 - accuracy: 0.9770
245/375 [==================>...........] - ETA: 1s - loss: 0.0729 - accuracy: 0.9770
251/375 [===================>..........] - ETA: 1s - loss: 0.0723 - accuracy: 0.9772
257/375 [===================>..........] - ETA: 1s - loss: 0.0718 - accuracy: 0.9773
264/375 [====================>.........] - ETA: 0s - loss: 0.0724 - accuracy: 0.9772
268/375 [====================>.........] - ETA: 0s - loss: 0.0722 - accuracy: 0.9772
274/375 [====================>.........] - ETA: 0s - loss: 0.0728 - accuracy: 0.9771
280/375 [=====================>........] - ETA: 0s - loss: 0.0732 - accuracy: 0.9770
286/375 [=====================>........] - ETA: 0s - loss: 0.0730 - accuracy: 0.9771
292/375 [======================>.......] - ETA: 0s - loss: 0.0733 - accuracy: 0.9769
298/375 [======================>.......] - ETA: 0s - loss: 0.0730 - accuracy: 0.9770
304/375 [=======================>......] - ETA: 0s - loss: 0.0728 - accuracy: 0.9772
311/375 [=======================>......] - ETA: 0s - loss: 0.0729 - accuracy: 0.9771
318/375 [========================>.....] - ETA: 0s - loss: 0.0728 - accuracy: 0.9771
324/375 [========================>.....] - ETA: 0s - loss: 0.0726 - accuracy: 0.9771
330/375 [=========================>....] - ETA: 0s - loss: 0.0726 - accuracy: 0.9772
337/375 [=========================>....] - ETA: 0s - loss: 0.0727 - accuracy: 0.9770
344/375 [==========================>...] - ETA: 0s - loss: 0.0729 - accuracy: 0.9770
351/375 [===========================>..] - ETA: 0s - loss: 0.0727 - accuracy: 0.9771
358/375 [===========================>..] - ETA: 0s - loss: 0.0730 - accuracy: 0.9771
364/375 [============================>.] - ETA: 0s - loss: 0.0732 - accuracy: 0.9771
370/375 [============================>.] - ETA: 0s - loss: 0.0736 - accuracy: 0.9769
375/375 [==============================] - 3s 9ms/step - loss: 0.0735 - accuracy: 0.9769

375/375 [==============================] - 4s 11ms/step - loss: 0.0735 - accuracy: 0.9769 - val_loss: 0.1475 - val_accuracy: 0.9628 - lr: 0.1000
Epoch 11/20

  1/375 [..............................] - ETA: 2s - loss: 0.0751 - accuracy: 0.9609
  8/375 [..............................] - ETA: 3s - loss: 0.0906 - accuracy: 0.9668
 14/375 [>.............................] - ETA: 3s - loss: 0.0790 - accuracy: 0.9738
 20/375 [>.............................] - ETA: 2s - loss: 0.0717 - accuracy: 0.9762
 26/375 [=>............................] - ETA: 2s - loss: 0.0791 - accuracy: 0.9742
 32/375 [=>............................] - ETA: 2s - loss: 0.0764 - accuracy: 0.9751
 39/375 [==>...........................] - ETA: 2s - loss: 0.0701 - accuracy: 0.9768
 45/375 [==>...........................] - ETA: 2s - loss: 0.0658 - accuracy: 0.9780
 50/375 [===>..........................] - ETA: 2s - loss: 0.0636 - accuracy: 0.9783
 57/375 [===>..........................] - ETA: 2s - loss: 0.0601 - accuracy: 0.9794
 63/375 [====>.........................] - ETA: 2s - loss: 0.0577 - accuracy: 0.9809
 69/375 [====>.........................] - ETA: 2s - loss: 0.0554 - accuracy: 0.9818
 75/375 [=====>........................] - ETA: 2s - loss: 0.0538 - accuracy: 0.9827
 80/375 [=====>........................] - ETA: 2s - loss: 0.0533 - accuracy: 0.9829
 86/375 [=====>........................] - ETA: 2s - loss: 0.0531 - accuracy: 0.9831
 92/375 [======>.......................] - ETA: 2s - loss: 0.0537 - accuracy: 0.9831
 97/375 [======>.......................] - ETA: 2s - loss: 0.0532 - accuracy: 0.9832
103/375 [=======>......................] - ETA: 2s - loss: 0.0519 - accuracy: 0.9835
108/375 [=======>......................] - ETA: 2s - loss: 0.0510 - accuracy: 0.9838
113/375 [========>.....................] - ETA: 2s - loss: 0.0516 - accuracy: 0.9836
118/375 [========>.....................] - ETA: 2s - loss: 0.0510 - accuracy: 0.9837
124/375 [========>.....................] - ETA: 2s - loss: 0.0519 - accuracy: 0.9837
129/375 [=========>....................] - ETA: 2s - loss: 0.0520 - accuracy: 0.9837
134/375 [=========>....................] - ETA: 2s - loss: 0.0513 - accuracy: 0.9840
140/375 [==========>...................] - ETA: 2s - loss: 0.0514 - accuracy: 0.9838
145/375 [==========>...................] - ETA: 2s - loss: 0.0505 - accuracy: 0.9842
150/375 [===========>..................] - ETA: 2s - loss: 0.0505 - accuracy: 0.9841
156/375 [===========>..................] - ETA: 2s - loss: 0.0507 - accuracy: 0.9839
161/375 [===========>..................] - ETA: 2s - loss: 0.0501 - accuracy: 0.9840
166/375 [============>.................] - ETA: 1s - loss: 0.0500 - accuracy: 0.9840
172/375 [============>.................] - ETA: 1s - loss: 0.0501 - accuracy: 0.9841
178/375 [=============>................] - ETA: 1s - loss: 0.0496 - accuracy: 0.9842
184/375 [=============>................] - ETA: 1s - loss: 0.0490 - accuracy: 0.9843
190/375 [==============>...............] - ETA: 1s - loss: 0.0485 - accuracy: 0.9844
196/375 [==============>...............] - ETA: 1s - loss: 0.0485 - accuracy: 0.9844
202/375 [===============>..............] - ETA: 1s - loss: 0.0482 - accuracy: 0.9844
208/375 [===============>..............] - ETA: 1s - loss: 0.0480 - accuracy: 0.9845
215/375 [================>.............] - ETA: 1s - loss: 0.0474 - accuracy: 0.9847
222/375 [================>.............] - ETA: 1s - loss: 0.0469 - accuracy: 0.9848
229/375 [=================>............] - ETA: 1s - loss: 0.0462 - accuracy: 0.9850
235/375 [=================>............] - ETA: 1s - loss: 0.0460 - accuracy: 0.9851
241/375 [==================>...........] - ETA: 1s - loss: 0.0455 - accuracy: 0.9852
247/375 [==================>...........] - ETA: 1s - loss: 0.0453 - accuracy: 0.9852
253/375 [===================>..........] - ETA: 1s - loss: 0.0448 - accuracy: 0.9853
259/375 [===================>..........] - ETA: 1s - loss: 0.0445 - accuracy: 0.9854
265/375 [====================>.........] - ETA: 1s - loss: 0.0440 - accuracy: 0.9856
271/375 [====================>.........] - ETA: 0s - loss: 0.0440 - accuracy: 0.9856
277/375 [=====================>........] - ETA: 0s - loss: 0.0440 - accuracy: 0.9857
284/375 [=====================>........] - ETA: 0s - loss: 0.0440 - accuracy: 0.9857
290/375 [======================>.......] - ETA: 0s - loss: 0.0438 - accuracy: 0.9857
296/375 [======================>.......] - ETA: 0s - loss: 0.0438 - accuracy: 0.9857
302/375 [=======================>......] - ETA: 0s - loss: 0.0439 - accuracy: 0.9857
309/375 [=======================>......] - ETA: 0s - loss: 0.0439 - accuracy: 0.9857
316/375 [========================>.....] - ETA: 0s - loss: 0.0440 - accuracy: 0.9857
322/375 [========================>.....] - ETA: 0s - loss: 0.0443 - accuracy: 0.9856
328/375 [=========================>....] - ETA: 0s - loss: 0.0443 - accuracy: 0.9856
334/375 [=========================>....] - ETA: 0s - loss: 0.0442 - accuracy: 0.9857
339/375 [==========================>...] - ETA: 0s - loss: 0.0442 - accuracy: 0.9857
344/375 [==========================>...] - ETA: 0s - loss: 0.0443 - accuracy: 0.9857
350/375 [===========================>..] - ETA: 0s - loss: 0.0441 - accuracy: 0.9857
355/375 [===========================>..] - ETA: 0s - loss: 0.0438 - accuracy: 0.9858
361/375 [===========================>..] - ETA: 0s - loss: 0.0440 - accuracy: 0.9858
367/375 [============================>.] - ETA: 0s - loss: 0.0439 - accuracy: 0.9858
373/375 [============================>.] - ETA: 0s - loss: 0.0438 - accuracy: 0.9859
375/375 [==============================] - 3s 9ms/step - loss: 0.0437 - accuracy: 0.9859

375/375 [==============================] - 4s 12ms/step - loss: 0.0437 - accuracy: 0.9859 - val_loss: 0.1090 - val_accuracy: 0.9701 - lr: 0.0100
Epoch 12/20

  1/375 [..............................] - ETA: 2s - loss: 0.0087 - accuracy: 1.0000
  7/375 [..............................] - ETA: 3s - loss: 0.0265 - accuracy: 0.9900
 13/375 [>.............................] - ETA: 3s - loss: 0.0271 - accuracy: 0.9904
 20/375 [>.............................] - ETA: 2s - loss: 0.0279 - accuracy: 0.9902
 27/375 [=>............................] - ETA: 2s - loss: 0.0281 - accuracy: 0.9907
 33/375 [=>............................] - ETA: 2s - loss: 0.0297 - accuracy: 0.9893
 39/375 [==>...........................] - ETA: 2s - loss: 0.0299 - accuracy: 0.9898
 44/375 [==>...........................] - ETA: 2s - loss: 0.0290 - accuracy: 0.9899
 50/375 [===>..........................] - ETA: 2s - loss: 0.0312 - accuracy: 0.9895
 56/375 [===>..........................] - ETA: 2s - loss: 0.0309 - accuracy: 0.9897
 62/375 [===>..........................] - ETA: 2s - loss: 0.0304 - accuracy: 0.9899
 69/375 [====>.........................] - ETA: 2s - loss: 0.0305 - accuracy: 0.9899
 75/375 [=====>........................] - ETA: 2s - loss: 0.0299 - accuracy: 0.9902
 81/375 [=====>........................] - ETA: 2s - loss: 0.0296 - accuracy: 0.9904
 87/375 [=====>........................] - ETA: 2s - loss: 0.0311 - accuracy: 0.9899
 93/375 [======>.......................] - ETA: 2s - loss: 0.0306 - accuracy: 0.9901
100/375 [=======>......................] - ETA: 2s - loss: 0.0316 - accuracy: 0.9898
107/375 [=======>......................] - ETA: 2s - loss: 0.0318 - accuracy: 0.9898
113/375 [========>.....................] - ETA: 2s - loss: 0.0321 - accuracy: 0.9900
119/375 [========>.....................] - ETA: 2s - loss: 0.0321 - accuracy: 0.9899
126/375 [=========>....................] - ETA: 2s - loss: 0.0325 - accuracy: 0.9899
132/375 [=========>....................] - ETA: 2s - loss: 0.0325 - accuracy: 0.9899
139/375 [==========>...................] - ETA: 2s - loss: 0.0324 - accuracy: 0.9901
146/375 [==========>...................] - ETA: 1s - loss: 0.0322 - accuracy: 0.9902
153/375 [===========>..................] - ETA: 1s - loss: 0.0318 - accuracy: 0.9903
158/375 [===========>..................] - ETA: 1s - loss: 0.0319 - accuracy: 0.9903
164/375 [============>.................] - ETA: 1s - loss: 0.0321 - accuracy: 0.9903
170/375 [============>.................] - ETA: 1s - loss: 0.0324 - accuracy: 0.9902
176/375 [=============>................] - ETA: 1s - loss: 0.0324 - accuracy: 0.9901
183/375 [=============>................] - ETA: 1s - loss: 0.0321 - accuracy: 0.9902
188/375 [==============>...............] - ETA: 1s - loss: 0.0321 - accuracy: 0.9902
194/375 [==============>...............] - ETA: 1s - loss: 0.0318 - accuracy: 0.9903
200/375 [===============>..............] - ETA: 1s - loss: 0.0316 - accuracy: 0.9904
206/375 [===============>..............] - ETA: 1s - loss: 0.0312 - accuracy: 0.9905
212/375 [===============>..............] - ETA: 1s - loss: 0.0314 - accuracy: 0.9904
218/375 [================>.............] - ETA: 1s - loss: 0.0311 - accuracy: 0.9905
224/375 [================>.............] - ETA: 1s - loss: 0.0312 - accuracy: 0.9905
230/375 [=================>............] - ETA: 1s - loss: 0.0311 - accuracy: 0.9905
236/375 [=================>............] - ETA: 1s - loss: 0.0308 - accuracy: 0.9905
242/375 [==================>...........] - ETA: 1s - loss: 0.0311 - accuracy: 0.9904
248/375 [==================>...........] - ETA: 1s - loss: 0.0312 - accuracy: 0.9904
254/375 [===================>..........] - ETA: 1s - loss: 0.0313 - accuracy: 0.9904
260/375 [===================>..........] - ETA: 0s - loss: 0.0312 - accuracy: 0.9904
265/375 [====================>.........] - ETA: 0s - loss: 0.0310 - accuracy: 0.9905
271/375 [====================>.........] - ETA: 0s - loss: 0.0311 - accuracy: 0.9905
277/375 [=====================>........] - ETA: 0s - loss: 0.0312 - accuracy: 0.9905
283/375 [=====================>........] - ETA: 0s - loss: 0.0310 - accuracy: 0.9906
289/375 [======================>.......] - ETA: 0s - loss: 0.0309 - accuracy: 0.9907
295/375 [======================>.......] - ETA: 0s - loss: 0.0313 - accuracy: 0.9906
301/375 [=======================>......] - ETA: 0s - loss: 0.0311 - accuracy: 0.9906
307/375 [=======================>......] - ETA: 0s - loss: 0.0310 - accuracy: 0.9907
313/375 [========================>.....] - ETA: 0s - loss: 0.0309 - accuracy: 0.9906
319/375 [========================>.....] - ETA: 0s - loss: 0.0313 - accuracy: 0.9905
325/375 [=========================>....] - ETA: 0s - loss: 0.0314 - accuracy: 0.9904
331/375 [=========================>....] - ETA: 0s - loss: 0.0311 - accuracy: 0.9905
337/375 [=========================>....] - ETA: 0s - loss: 0.0310 - accuracy: 0.9905
343/375 [==========================>...] - ETA: 0s - loss: 0.0310 - accuracy: 0.9905
349/375 [==========================>...] - ETA: 0s - loss: 0.0309 - accuracy: 0.9905
355/375 [===========================>..] - ETA: 0s - loss: 0.0310 - accuracy: 0.9905
361/375 [===========================>..] - ETA: 0s - loss: 0.0306 - accuracy: 0.9906
367/375 [============================>.] - ETA: 0s - loss: 0.0306 - accuracy: 0.9906
373/375 [============================>.] - ETA: 0s - loss: 0.0307 - accuracy: 0.9905
375/375 [==============================] - 3s 9ms/step - loss: 0.0306 - accuracy: 0.9906

375/375 [==============================] - 4s 11ms/step - loss: 0.0306 - accuracy: 0.9906 - val_loss: 0.1081 - val_accuracy: 0.9712 - lr: 0.0100
Epoch 13/20

  1/375 [..............................] - ETA: 2s - loss: 0.0581 - accuracy: 0.9688
  7/375 [..............................] - ETA: 3s - loss: 0.0329 - accuracy: 0.9900
 13/375 [>.............................] - ETA: 3s - loss: 0.0299 - accuracy: 0.9892
 19/375 [>.............................] - ETA: 3s - loss: 0.0287 - accuracy: 0.9901
 26/375 [=>............................] - ETA: 2s - loss: 0.0263 - accuracy: 0.9913
 30/375 [=>............................] - ETA: 3s - loss: 0.0273 - accuracy: 0.9911
 36/375 [=>............................] - ETA: 3s - loss: 0.0314 - accuracy: 0.9902
 42/375 [==>...........................] - ETA: 3s - loss: 0.0330 - accuracy: 0.9903
 48/375 [==>...........................] - ETA: 2s - loss: 0.0319 - accuracy: 0.9904
 54/375 [===>..........................] - ETA: 2s - loss: 0.0311 - accuracy: 0.9902
 60/375 [===>..........................] - ETA: 2s - loss: 0.0307 - accuracy: 0.9905
 66/375 [====>.........................] - ETA: 2s - loss: 0.0301 - accuracy: 0.9908
 72/375 [====>.........................] - ETA: 2s - loss: 0.0293 - accuracy: 0.9908
 78/375 [=====>........................] - ETA: 2s - loss: 0.0295 - accuracy: 0.9904
 84/375 [=====>........................] - ETA: 2s - loss: 0.0287 - accuracy: 0.9904
 90/375 [======>.......................] - ETA: 2s - loss: 0.0286 - accuracy: 0.9903
 96/375 [======>.......................] - ETA: 2s - loss: 0.0278 - accuracy: 0.9906
103/375 [=======>......................] - ETA: 2s - loss: 0.0278 - accuracy: 0.9907
109/375 [=======>......................] - ETA: 2s - loss: 0.0277 - accuracy: 0.9908
115/375 [========>.....................] - ETA: 2s - loss: 0.0293 - accuracy: 0.9905
122/375 [========>.....................] - ETA: 2s - loss: 0.0288 - accuracy: 0.9906
128/375 [=========>....................] - ETA: 2s - loss: 0.0282 - accuracy: 0.9909
134/375 [=========>....................] - ETA: 2s - loss: 0.0280 - accuracy: 0.9911
140/375 [==========>...................] - ETA: 2s - loss: 0.0285 - accuracy: 0.9908
146/375 [==========>...................] - ETA: 2s - loss: 0.0288 - accuracy: 0.9907
152/375 [===========>..................] - ETA: 1s - loss: 0.0289 - accuracy: 0.9906
157/375 [===========>..................] - ETA: 1s - loss: 0.0288 - accuracy: 0.9907
163/375 [============>.................] - ETA: 1s - loss: 0.0284 - accuracy: 0.9909
168/375 [============>.................] - ETA: 1s - loss: 0.0280 - accuracy: 0.9911
174/375 [============>.................] - ETA: 1s - loss: 0.0284 - accuracy: 0.9910
180/375 [=============>................] - ETA: 1s - loss: 0.0282 - accuracy: 0.9911
185/375 [=============>................] - ETA: 1s - loss: 0.0288 - accuracy: 0.9909
191/375 [==============>...............] - ETA: 1s - loss: 0.0285 - accuracy: 0.9910
198/375 [==============>...............] - ETA: 1s - loss: 0.0283 - accuracy: 0.9911
203/375 [===============>..............] - ETA: 1s - loss: 0.0283 - accuracy: 0.9911
205/375 [===============>..............] - ETA: 1s - loss: 0.0282 - accuracy: 0.9912
211/375 [===============>..............] - ETA: 1s - loss: 0.0281 - accuracy: 0.9912
217/375 [================>.............] - ETA: 1s - loss: 0.0283 - accuracy: 0.9912
223/375 [================>.............] - ETA: 1s - loss: 0.0281 - accuracy: 0.9913
229/375 [=================>............] - ETA: 1s - loss: 0.0279 - accuracy: 0.9913
235/375 [=================>............] - ETA: 1s - loss: 0.0277 - accuracy: 0.9914
239/375 [==================>...........] - ETA: 1s - loss: 0.0278 - accuracy: 0.9914
245/375 [==================>...........] - ETA: 1s - loss: 0.0278 - accuracy: 0.9914
251/375 [===================>..........] - ETA: 1s - loss: 0.0280 - accuracy: 0.9913
257/375 [===================>..........] - ETA: 1s - loss: 0.0277 - accuracy: 0.9914
263/375 [====================>.........] - ETA: 1s - loss: 0.0277 - accuracy: 0.9914
269/375 [====================>.........] - ETA: 0s - loss: 0.0273 - accuracy: 0.9915
276/375 [=====================>........] - ETA: 0s - loss: 0.0272 - accuracy: 0.9915
282/375 [=====================>........] - ETA: 0s - loss: 0.0272 - accuracy: 0.9916
288/375 [======================>.......] - ETA: 0s - loss: 0.0273 - accuracy: 0.9916
294/375 [======================>.......] - ETA: 0s - loss: 0.0273 - accuracy: 0.9916
300/375 [=======================>......] - ETA: 0s - loss: 0.0273 - accuracy: 0.9916
306/375 [=======================>......] - ETA: 0s - loss: 0.0274 - accuracy: 0.9916
312/375 [=======================>......] - ETA: 0s - loss: 0.0274 - accuracy: 0.9916
318/375 [========================>.....] - ETA: 0s - loss: 0.0273 - accuracy: 0.9917
324/375 [========================>.....] - ETA: 0s - loss: 0.0273 - accuracy: 0.9917
330/375 [=========================>....] - ETA: 0s - loss: 0.0271 - accuracy: 0.9917
336/375 [=========================>....] - ETA: 0s - loss: 0.0271 - accuracy: 0.9917
342/375 [==========================>...] - ETA: 0s - loss: 0.0274 - accuracy: 0.9916
348/375 [==========================>...] - ETA: 0s - loss: 0.0272 - accuracy: 0.9916
354/375 [===========================>..] - ETA: 0s - loss: 0.0274 - accuracy: 0.9916
360/375 [===========================>..] - ETA: 0s - loss: 0.0274 - accuracy: 0.9917
366/375 [============================>.] - ETA: 0s - loss: 0.0272 - accuracy: 0.9918
372/375 [============================>.] - ETA: 0s - loss: 0.0273 - accuracy: 0.9918
375/375 [==============================] - 3s 9ms/step - loss: 0.0273 - accuracy: 0.9918

375/375 [==============================] - 4s 12ms/step - loss: 0.0273 - accuracy: 0.9918 - val_loss: 0.1077 - val_accuracy: 0.9716 - lr: 0.0100
Epoch 14/20

  1/375 [..............................] - ETA: 3s - loss: 0.0067 - accuracy: 1.0000
  8/375 [..............................] - ETA: 2s - loss: 0.0278 - accuracy: 0.9912
 14/375 [>.............................] - ETA: 3s - loss: 0.0250 - accuracy: 0.9927
 20/375 [>.............................] - ETA: 3s - loss: 0.0279 - accuracy: 0.9918
 26/375 [=>............................] - ETA: 2s - loss: 0.0264 - accuracy: 0.9919
 32/375 [=>............................] - ETA: 2s - loss: 0.0258 - accuracy: 0.9924
 39/375 [==>...........................] - ETA: 2s - loss: 0.0242 - accuracy: 0.9930
 45/375 [==>...........................] - ETA: 2s - loss: 0.0258 - accuracy: 0.9927
 51/375 [===>..........................] - ETA: 2s - loss: 0.0239 - accuracy: 0.9933
 58/375 [===>..........................] - ETA: 2s - loss: 0.0239 - accuracy: 0.9930
 64/375 [====>.........................] - ETA: 2s - loss: 0.0239 - accuracy: 0.9932
 70/375 [====>.........................] - ETA: 2s - loss: 0.0254 - accuracy: 0.9927
 77/375 [=====>........................] - ETA: 2s - loss: 0.0250 - accuracy: 0.9928
 83/375 [=====>........................] - ETA: 2s - loss: 0.0254 - accuracy: 0.9926
 89/375 [======>.......................] - ETA: 2s - loss: 0.0252 - accuracy: 0.9928
 95/375 [======>.......................] - ETA: 2s - loss: 0.0249 - accuracy: 0.9928
101/375 [=======>......................] - ETA: 2s - loss: 0.0243 - accuracy: 0.9930
107/375 [=======>......................] - ETA: 2s - loss: 0.0241 - accuracy: 0.9931
113/375 [========>.....................] - ETA: 2s - loss: 0.0246 - accuracy: 0.9927
119/375 [========>.....................] - ETA: 2s - loss: 0.0246 - accuracy: 0.9928
125/375 [=========>....................] - ETA: 2s - loss: 0.0248 - accuracy: 0.9927
131/375 [=========>....................] - ETA: 2s - loss: 0.0247 - accuracy: 0.9927
137/375 [=========>....................] - ETA: 2s - loss: 0.0247 - accuracy: 0.9926
143/375 [==========>...................] - ETA: 2s - loss: 0.0246 - accuracy: 0.9926
149/375 [==========>...................] - ETA: 1s - loss: 0.0249 - accuracy: 0.9924
155/375 [===========>..................] - ETA: 1s - loss: 0.0252 - accuracy: 0.9923
161/375 [===========>..................] - ETA: 1s - loss: 0.0250 - accuracy: 0.9923
167/375 [============>.................] - ETA: 1s - loss: 0.0248 - accuracy: 0.9925
173/375 [============>.................] - ETA: 1s - loss: 0.0249 - accuracy: 0.9924
179/375 [=============>................] - ETA: 1s - loss: 0.0247 - accuracy: 0.9924
185/375 [=============>................] - ETA: 1s - loss: 0.0248 - accuracy: 0.9923
192/375 [==============>...............] - ETA: 1s - loss: 0.0244 - accuracy: 0.9924
198/375 [==============>...............] - ETA: 1s - loss: 0.0241 - accuracy: 0.9926
204/375 [===============>..............] - ETA: 1s - loss: 0.0239 - accuracy: 0.9926
210/375 [===============>..............] - ETA: 1s - loss: 0.0239 - accuracy: 0.9926
216/375 [================>.............] - ETA: 1s - loss: 0.0237 - accuracy: 0.9927
222/375 [================>.............] - ETA: 1s - loss: 0.0238 - accuracy: 0.9927
228/375 [=================>............] - ETA: 1s - loss: 0.0239 - accuracy: 0.9927
234/375 [=================>............] - ETA: 1s - loss: 0.0246 - accuracy: 0.9926
241/375 [==================>...........] - ETA: 1s - loss: 0.0244 - accuracy: 0.9926
247/375 [==================>...........] - ETA: 1s - loss: 0.0246 - accuracy: 0.9926
254/375 [===================>..........] - ETA: 1s - loss: 0.0245 - accuracy: 0.9926
260/375 [===================>..........] - ETA: 1s - loss: 0.0246 - accuracy: 0.9926
266/375 [====================>.........] - ETA: 0s - loss: 0.0246 - accuracy: 0.9927
272/375 [====================>.........] - ETA: 0s - loss: 0.0248 - accuracy: 0.9925
278/375 [=====================>........] - ETA: 0s - loss: 0.0247 - accuracy: 0.9926
285/375 [=====================>........] - ETA: 0s - loss: 0.0246 - accuracy: 0.9926
291/375 [======================>.......] - ETA: 0s - loss: 0.0245 - accuracy: 0.9926
297/375 [======================>.......] - ETA: 0s - loss: 0.0244 - accuracy: 0.9926
303/375 [=======================>......] - ETA: 0s - loss: 0.0246 - accuracy: 0.9926
310/375 [=======================>......] - ETA: 0s - loss: 0.0244 - accuracy: 0.9926
316/375 [========================>.....] - ETA: 0s - loss: 0.0244 - accuracy: 0.9927
323/375 [========================>.....] - ETA: 0s - loss: 0.0244 - accuracy: 0.9927
329/375 [=========================>....] - ETA: 0s - loss: 0.0250 - accuracy: 0.9926
335/375 [=========================>....] - ETA: 0s - loss: 0.0250 - accuracy: 0.9927
342/375 [==========================>...] - ETA: 0s - loss: 0.0249 - accuracy: 0.9927
349/375 [==========================>...] - ETA: 0s - loss: 0.0249 - accuracy: 0.9927
355/375 [===========================>..] - ETA: 0s - loss: 0.0247 - accuracy: 0.9928
361/375 [===========================>..] - ETA: 0s - loss: 0.0249 - accuracy: 0.9927
368/375 [============================>.] - ETA: 0s - loss: 0.0247 - accuracy: 0.9928
374/375 [============================>.] - ETA: 0s - loss: 0.0249 - accuracy: 0.9927
375/375 [==============================] - 3s 9ms/step - loss: 0.0249 - accuracy: 0.9927

375/375 [==============================] - 4s 11ms/step - loss: 0.0249 - accuracy: 0.9927 - val_loss: 0.1080 - val_accuracy: 0.9722 - lr: 0.0100
Epoch 15/20

  1/375 [..............................] - ETA: 2s - loss: 0.0098 - accuracy: 1.0000
  7/375 [..............................] - ETA: 3s - loss: 0.0172 - accuracy: 0.9967
 13/375 [>.............................] - ETA: 3s - loss: 0.0225 - accuracy: 0.9928
 20/375 [>.............................] - ETA: 2s - loss: 0.0234 - accuracy: 0.9926
 27/375 [=>............................] - ETA: 2s - loss: 0.0240 - accuracy: 0.9916
 34/375 [=>............................] - ETA: 2s - loss: 0.0245 - accuracy: 0.9924
 40/375 [==>...........................] - ETA: 2s - loss: 0.0250 - accuracy: 0.9926
 45/375 [==>...........................] - ETA: 2s - loss: 0.0239 - accuracy: 0.9931
 51/375 [===>..........................] - ETA: 2s - loss: 0.0226 - accuracy: 0.9933
 57/375 [===>..........................] - ETA: 2s - loss: 0.0225 - accuracy: 0.9933
 63/375 [====>.........................] - ETA: 2s - loss: 0.0224 - accuracy: 0.9933
 69/375 [====>.........................] - ETA: 2s - loss: 0.0227 - accuracy: 0.9933
 75/375 [=====>........................] - ETA: 2s - loss: 0.0231 - accuracy: 0.9936
 81/375 [=====>........................] - ETA: 2s - loss: 0.0222 - accuracy: 0.9940
 87/375 [=====>........................] - ETA: 2s - loss: 0.0220 - accuracy: 0.9943
 93/375 [======>.......................] - ETA: 2s - loss: 0.0222 - accuracy: 0.9940
 99/375 [======>.......................] - ETA: 2s - loss: 0.0217 - accuracy: 0.9942
105/375 [=======>......................] - ETA: 2s - loss: 0.0213 - accuracy: 0.9942
111/375 [=======>......................] - ETA: 2s - loss: 0.0209 - accuracy: 0.9943
117/375 [========>.....................] - ETA: 2s - loss: 0.0212 - accuracy: 0.9942
123/375 [========>.....................] - ETA: 2s - loss: 0.0216 - accuracy: 0.9941
129/375 [=========>....................] - ETA: 2s - loss: 0.0218 - accuracy: 0.9940
135/375 [=========>....................] - ETA: 2s - loss: 0.0215 - accuracy: 0.9939
140/375 [==========>...................] - ETA: 2s - loss: 0.0220 - accuracy: 0.9937
146/375 [==========>...................] - ETA: 2s - loss: 0.0215 - accuracy: 0.9939
152/375 [===========>..................] - ETA: 1s - loss: 0.0215 - accuracy: 0.9939
159/375 [===========>..................] - ETA: 1s - loss: 0.0221 - accuracy: 0.9938
165/375 [============>.................] - ETA: 1s - loss: 0.0218 - accuracy: 0.9939
171/375 [============>.................] - ETA: 1s - loss: 0.0217 - accuracy: 0.9939
177/375 [=============>................] - ETA: 1s - loss: 0.0214 - accuracy: 0.9940
183/375 [=============>................] - ETA: 1s - loss: 0.0216 - accuracy: 0.9938
189/375 [==============>...............] - ETA: 1s - loss: 0.0214 - accuracy: 0.9939
196/375 [==============>...............] - ETA: 1s - loss: 0.0214 - accuracy: 0.9939
202/375 [===============>..............] - ETA: 1s - loss: 0.0213 - accuracy: 0.9939
208/375 [===============>..............] - ETA: 1s - loss: 0.0211 - accuracy: 0.9940
214/375 [================>.............] - ETA: 1s - loss: 0.0213 - accuracy: 0.9939
220/375 [================>.............] - ETA: 1s - loss: 0.0217 - accuracy: 0.9939
226/375 [=================>............] - ETA: 1s - loss: 0.0218 - accuracy: 0.9938
232/375 [=================>............] - ETA: 1s - loss: 0.0222 - accuracy: 0.9938
238/375 [==================>...........] - ETA: 1s - loss: 0.0223 - accuracy: 0.9936
244/375 [==================>...........] - ETA: 1s - loss: 0.0224 - accuracy: 0.9937
250/375 [===================>..........] - ETA: 1s - loss: 0.0222 - accuracy: 0.9937
256/375 [===================>..........] - ETA: 1s - loss: 0.0225 - accuracy: 0.9937
262/375 [===================>..........] - ETA: 1s - loss: 0.0226 - accuracy: 0.9936
268/375 [====================>.........] - ETA: 0s - loss: 0.0225 - accuracy: 0.9936
274/375 [====================>.........] - ETA: 0s - loss: 0.0223 - accuracy: 0.9936
280/375 [=====================>........] - ETA: 0s - loss: 0.0224 - accuracy: 0.9936
286/375 [=====================>........] - ETA: 0s - loss: 0.0224 - accuracy: 0.9936
292/375 [======================>.......] - ETA: 0s - loss: 0.0222 - accuracy: 0.9937
298/375 [======================>.......] - ETA: 0s - loss: 0.0222 - accuracy: 0.9937
304/375 [=======================>......] - ETA: 0s - loss: 0.0221 - accuracy: 0.9937
310/375 [=======================>......] - ETA: 0s - loss: 0.0222 - accuracy: 0.9937
316/375 [========================>.....] - ETA: 0s - loss: 0.0221 - accuracy: 0.9937
322/375 [========================>.....] - ETA: 0s - loss: 0.0222 - accuracy: 0.9937
328/375 [=========================>....] - ETA: 0s - loss: 0.0220 - accuracy: 0.9937
334/375 [=========================>....] - ETA: 0s - loss: 0.0220 - accuracy: 0.9937
340/375 [==========================>...] - ETA: 0s - loss: 0.0221 - accuracy: 0.9937
346/375 [==========================>...] - ETA: 0s - loss: 0.0221 - accuracy: 0.9937
351/375 [===========================>..] - ETA: 0s - loss: 0.0221 - accuracy: 0.9937
357/375 [===========================>..] - ETA: 0s - loss: 0.0220 - accuracy: 0.9937
363/375 [============================>.] - ETA: 0s - loss: 0.0219 - accuracy: 0.9938
369/375 [============================>.] - ETA: 0s - loss: 0.0219 - accuracy: 0.9937
375/375 [==============================] - 3s 9ms/step - loss: 0.0220 - accuracy: 0.9937

375/375 [==============================] - 4s 11ms/step - loss: 0.0220 - accuracy: 0.9937 - val_loss: 0.1078 - val_accuracy: 0.9724 - lr: 1.0000e-03
Epoch 16/20

  1/375 [..............................] - ETA: 3s - loss: 0.0199 - accuracy: 0.9922
  7/375 [..............................] - ETA: 3s - loss: 0.0178 - accuracy: 0.9933
 14/375 [>.............................] - ETA: 3s - loss: 0.0157 - accuracy: 0.9950
 20/375 [>.............................] - ETA: 2s - loss: 0.0158 - accuracy: 0.9953
 26/375 [=>............................] - ETA: 2s - loss: 0.0165 - accuracy: 0.9949
 33/375 [=>............................] - ETA: 2s - loss: 0.0192 - accuracy: 0.9943
 38/375 [==>...........................] - ETA: 2s - loss: 0.0188 - accuracy: 0.9944
 44/375 [==>...........................] - ETA: 2s - loss: 0.0191 - accuracy: 0.9940
 50/375 [===>..........................] - ETA: 2s - loss: 0.0194 - accuracy: 0.9939
 56/375 [===>..........................] - ETA: 2s - loss: 0.0192 - accuracy: 0.9941
 62/375 [===>..........................] - ETA: 2s - loss: 0.0197 - accuracy: 0.9941
 68/375 [====>.........................] - ETA: 2s - loss: 0.0198 - accuracy: 0.9939
 73/375 [====>.........................] - ETA: 2s - loss: 0.0195 - accuracy: 0.9941
 78/375 [=====>........................] - ETA: 2s - loss: 0.0199 - accuracy: 0.9939
 84/375 [=====>........................] - ETA: 2s - loss: 0.0204 - accuracy: 0.9939
 90/375 [======>.......................] - ETA: 2s - loss: 0.0218 - accuracy: 0.9937
 96/375 [======>.......................] - ETA: 2s - loss: 0.0211 - accuracy: 0.9938
102/375 [=======>......................] - ETA: 2s - loss: 0.0212 - accuracy: 0.9937
108/375 [=======>......................] - ETA: 2s - loss: 0.0210 - accuracy: 0.9939
113/375 [========>.....................] - ETA: 2s - loss: 0.0216 - accuracy: 0.9937
119/375 [========>.....................] - ETA: 2s - loss: 0.0217 - accuracy: 0.9936
125/375 [=========>....................] - ETA: 2s - loss: 0.0216 - accuracy: 0.9936
131/375 [=========>....................] - ETA: 2s - loss: 0.0214 - accuracy: 0.9937
137/375 [=========>....................] - ETA: 2s - loss: 0.0214 - accuracy: 0.9936
143/375 [==========>...................] - ETA: 2s - loss: 0.0212 - accuracy: 0.9936
149/375 [==========>...................] - ETA: 2s - loss: 0.0209 - accuracy: 0.9937
155/375 [===========>..................] - ETA: 2s - loss: 0.0212 - accuracy: 0.9936
161/375 [===========>..................] - ETA: 1s - loss: 0.0213 - accuracy: 0.9936
167/375 [============>.................] - ETA: 1s - loss: 0.0216 - accuracy: 0.9937
173/375 [============>.................] - ETA: 1s - loss: 0.0216 - accuracy: 0.9938
179/375 [=============>................] - ETA: 1s - loss: 0.0217 - accuracy: 0.9938
185/375 [=============>................] - ETA: 1s - loss: 0.0219 - accuracy: 0.9937
191/375 [==============>...............] - ETA: 1s - loss: 0.0218 - accuracy: 0.9937
197/375 [==============>...............] - ETA: 1s - loss: 0.0218 - accuracy: 0.9939
200/375 [===============>..............] - ETA: 1s - loss: 0.0216 - accuracy: 0.9939
206/375 [===============>..............] - ETA: 1s - loss: 0.0213 - accuracy: 0.9941
211/375 [===============>..............] - ETA: 1s - loss: 0.0212 - accuracy: 0.9941
217/375 [================>.............] - ETA: 1s - loss: 0.0214 - accuracy: 0.9941
223/375 [================>.............] - ETA: 1s - loss: 0.0211 - accuracy: 0.9941
229/375 [=================>............] - ETA: 1s - loss: 0.0212 - accuracy: 0.9940
235/375 [=================>............] - ETA: 1s - loss: 0.0215 - accuracy: 0.9939
241/375 [==================>...........] - ETA: 1s - loss: 0.0214 - accuracy: 0.9940
247/375 [==================>...........] - ETA: 1s - loss: 0.0217 - accuracy: 0.9939
253/375 [===================>..........] - ETA: 1s - loss: 0.0215 - accuracy: 0.9939
259/375 [===================>..........] - ETA: 1s - loss: 0.0214 - accuracy: 0.9940
265/375 [====================>.........] - ETA: 1s - loss: 0.0213 - accuracy: 0.9940
271/375 [====================>.........] - ETA: 0s - loss: 0.0212 - accuracy: 0.9940
277/375 [=====================>........] - ETA: 0s - loss: 0.0214 - accuracy: 0.9939
283/375 [=====================>........] - ETA: 0s - loss: 0.0214 - accuracy: 0.9940
289/375 [======================>.......] - ETA: 0s - loss: 0.0214 - accuracy: 0.9940
295/375 [======================>.......] - ETA: 0s - loss: 0.0214 - accuracy: 0.9940
301/375 [=======================>......] - ETA: 0s - loss: 0.0213 - accuracy: 0.9941
307/375 [=======================>......] - ETA: 0s - loss: 0.0214 - accuracy: 0.9940
313/375 [========================>.....] - ETA: 0s - loss: 0.0214 - accuracy: 0.9940
319/375 [========================>.....] - ETA: 0s - loss: 0.0213 - accuracy: 0.9941
325/375 [=========================>....] - ETA: 0s - loss: 0.0214 - accuracy: 0.9940
331/375 [=========================>....] - ETA: 0s - loss: 0.0214 - accuracy: 0.9941
337/375 [=========================>....] - ETA: 0s - loss: 0.0214 - accuracy: 0.9941
343/375 [==========================>...] - ETA: 0s - loss: 0.0214 - accuracy: 0.9941
349/375 [==========================>...] - ETA: 0s - loss: 0.0214 - accuracy: 0.9940
355/375 [===========================>..] - ETA: 0s - loss: 0.0214 - accuracy: 0.9940
360/375 [===========================>..] - ETA: 0s - loss: 0.0217 - accuracy: 0.9940
366/375 [============================>.] - ETA: 0s - loss: 0.0217 - accuracy: 0.9940
372/375 [============================>.] - ETA: 0s - loss: 0.0215 - accuracy: 0.9941
375/375 [==============================] - 3s 9ms/step - loss: 0.0216 - accuracy: 0.9941

375/375 [==============================] - 4s 11ms/step - loss: 0.0216 - accuracy: 0.9941 - val_loss: 0.1079 - val_accuracy: 0.9724 - lr: 1.0000e-04
Epoch 17/20

  1/375 [..............................] - ETA: 2s - loss: 0.0097 - accuracy: 1.0000
  7/375 [..............................] - ETA: 3s - loss: 0.0110 - accuracy: 0.9989
 13/375 [>.............................] - ETA: 3s - loss: 0.0190 - accuracy: 0.9952
 20/375 [>.............................] - ETA: 2s - loss: 0.0211 - accuracy: 0.9938
 26/375 [=>............................] - ETA: 3s - loss: 0.0210 - accuracy: 0.9931
 31/375 [=>............................] - ETA: 3s - loss: 0.0210 - accuracy: 0.9929
 37/375 [=>............................] - ETA: 3s - loss: 0.0207 - accuracy: 0.9930
 43/375 [==>...........................] - ETA: 2s - loss: 0.0233 - accuracy: 0.9926
 49/375 [==>...........................] - ETA: 2s - loss: 0.0217 - accuracy: 0.9933
 54/375 [===>..........................] - ETA: 2s - loss: 0.0238 - accuracy: 0.9932
 60/375 [===>..........................] - ETA: 2s - loss: 0.0238 - accuracy: 0.9931
 66/375 [====>.........................] - ETA: 2s - loss: 0.0233 - accuracy: 0.9935
 72/375 [====>.........................] - ETA: 2s - loss: 0.0226 - accuracy: 0.9938
 78/375 [=====>........................] - ETA: 2s - loss: 0.0232 - accuracy: 0.9936
 84/375 [=====>........................] - ETA: 2s - loss: 0.0235 - accuracy: 0.9934
 90/375 [======>.......................] - ETA: 2s - loss: 0.0226 - accuracy: 0.9938
 96/375 [======>.......................] - ETA: 2s - loss: 0.0225 - accuracy: 0.9941
102/375 [=======>......................] - ETA: 2s - loss: 0.0223 - accuracy: 0.9940
108/375 [=======>......................] - ETA: 2s - loss: 0.0219 - accuracy: 0.9941
113/375 [========>.....................] - ETA: 2s - loss: 0.0223 - accuracy: 0.9939
119/375 [========>.....................] - ETA: 2s - loss: 0.0221 - accuracy: 0.9940
125/375 [=========>....................] - ETA: 2s - loss: 0.0221 - accuracy: 0.9939
131/375 [=========>....................] - ETA: 2s - loss: 0.0221 - accuracy: 0.9939
137/375 [=========>....................] - ETA: 2s - loss: 0.0218 - accuracy: 0.9940
143/375 [==========>...................] - ETA: 2s - loss: 0.0217 - accuracy: 0.9940
149/375 [==========>...................] - ETA: 2s - loss: 0.0217 - accuracy: 0.9939
155/375 [===========>..................] - ETA: 1s - loss: 0.0220 - accuracy: 0.9938
161/375 [===========>..................] - ETA: 1s - loss: 0.0223 - accuracy: 0.9937
166/375 [============>.................] - ETA: 1s - loss: 0.0221 - accuracy: 0.9939
171/375 [============>.................] - ETA: 1s - loss: 0.0218 - accuracy: 0.9940
176/375 [=============>................] - ETA: 1s - loss: 0.0218 - accuracy: 0.9940
182/375 [=============>................] - ETA: 1s - loss: 0.0216 - accuracy: 0.9941
188/375 [==============>...............] - ETA: 1s - loss: 0.0213 - accuracy: 0.9942
194/375 [==============>...............] - ETA: 1s - loss: 0.0212 - accuracy: 0.9942
201/375 [===============>..............] - ETA: 1s - loss: 0.0209 - accuracy: 0.9944
208/375 [===============>..............] - ETA: 1s - loss: 0.0208 - accuracy: 0.9944
214/375 [================>.............] - ETA: 1s - loss: 0.0207 - accuracy: 0.9943
220/375 [================>.............] - ETA: 1s - loss: 0.0207 - accuracy: 0.9944
226/375 [=================>............] - ETA: 1s - loss: 0.0206 - accuracy: 0.9945
232/375 [=================>............] - ETA: 1s - loss: 0.0205 - accuracy: 0.9945
238/375 [==================>...........] - ETA: 1s - loss: 0.0206 - accuracy: 0.9945
244/375 [==================>...........] - ETA: 1s - loss: 0.0204 - accuracy: 0.9946
250/375 [===================>..........] - ETA: 1s - loss: 0.0203 - accuracy: 0.9946
256/375 [===================>..........] - ETA: 1s - loss: 0.0203 - accuracy: 0.9945
262/375 [===================>..........] - ETA: 1s - loss: 0.0204 - accuracy: 0.9945
268/375 [====================>.........] - ETA: 0s - loss: 0.0205 - accuracy: 0.9945
274/375 [====================>.........] - ETA: 0s - loss: 0.0204 - accuracy: 0.9946
280/375 [=====================>........] - ETA: 0s - loss: 0.0205 - accuracy: 0.9945
286/375 [=====================>........] - ETA: 0s - loss: 0.0206 - accuracy: 0.9945
292/375 [======================>.......] - ETA: 0s - loss: 0.0208 - accuracy: 0.9944
298/375 [======================>.......] - ETA: 0s - loss: 0.0211 - accuracy: 0.9944
304/375 [=======================>......] - ETA: 0s - loss: 0.0212 - accuracy: 0.9943
311/375 [=======================>......] - ETA: 0s - loss: 0.0217 - accuracy: 0.9942
318/375 [========================>.....] - ETA: 0s - loss: 0.0216 - accuracy: 0.9943
324/375 [========================>.....] - ETA: 0s - loss: 0.0214 - accuracy: 0.9943
330/375 [=========================>....] - ETA: 0s - loss: 0.0214 - accuracy: 0.9942
336/375 [=========================>....] - ETA: 0s - loss: 0.0215 - accuracy: 0.9942
341/375 [==========================>...] - ETA: 0s - loss: 0.0217 - accuracy: 0.9942
347/375 [==========================>...] - ETA: 0s - loss: 0.0217 - accuracy: 0.9940
353/375 [===========================>..] - ETA: 0s - loss: 0.0217 - accuracy: 0.9941
359/375 [===========================>..] - ETA: 0s - loss: 0.0215 - accuracy: 0.9941
365/375 [============================>.] - ETA: 0s - loss: 0.0214 - accuracy: 0.9941
371/375 [============================>.] - ETA: 0s - loss: 0.0214 - accuracy: 0.9941
375/375 [==============================] - 3s 9ms/step - loss: 0.0215 - accuracy: 0.9941

375/375 [==============================] - 4s 12ms/step - loss: 0.0215 - accuracy: 0.9941 - val_loss: 0.1079 - val_accuracy: 0.9724 - lr: 1.0000e-05
Epoch 18/20

  1/375 [..............................] - ETA: 3s - loss: 0.0182 - accuracy: 0.9922
  8/375 [..............................] - ETA: 2s - loss: 0.0221 - accuracy: 0.9941
 14/375 [>.............................] - ETA: 3s - loss: 0.0208 - accuracy: 0.9950
 21/375 [>.............................] - ETA: 2s - loss: 0.0215 - accuracy: 0.9933
 28/375 [=>............................] - ETA: 2s - loss: 0.0205 - accuracy: 0.9936
 35/375 [=>............................] - ETA: 2s - loss: 0.0220 - accuracy: 0.9933
 41/375 [==>...........................] - ETA: 2s - loss: 0.0223 - accuracy: 0.9931
 48/375 [==>...........................] - ETA: 2s - loss: 0.0232 - accuracy: 0.9935
 54/375 [===>..........................] - ETA: 2s - loss: 0.0219 - accuracy: 0.9941
 60/375 [===>..........................] - ETA: 2s - loss: 0.0226 - accuracy: 0.9934
 66/375 [====>.........................] - ETA: 2s - loss: 0.0228 - accuracy: 0.9933
 72/375 [====>.........................] - ETA: 2s - loss: 0.0234 - accuracy: 0.9932
 78/375 [=====>........................] - ETA: 2s - loss: 0.0236 - accuracy: 0.9933
 84/375 [=====>........................] - ETA: 2s - loss: 0.0250 - accuracy: 0.9930
 91/375 [======>.......................] - ETA: 2s - loss: 0.0246 - accuracy: 0.9934
 97/375 [======>.......................] - ETA: 2s - loss: 0.0238 - accuracy: 0.9936
103/375 [=======>......................] - ETA: 2s - loss: 0.0237 - accuracy: 0.9936
109/375 [=======>......................] - ETA: 2s - loss: 0.0240 - accuracy: 0.9935
115/375 [========>.....................] - ETA: 2s - loss: 0.0233 - accuracy: 0.9936
121/375 [========>.....................] - ETA: 2s - loss: 0.0236 - accuracy: 0.9935
126/375 [=========>....................] - ETA: 2s - loss: 0.0234 - accuracy: 0.9936
132/375 [=========>....................] - ETA: 2s - loss: 0.0228 - accuracy: 0.9937
137/375 [=========>....................] - ETA: 2s - loss: 0.0229 - accuracy: 0.9937
142/375 [==========>...................] - ETA: 2s - loss: 0.0228 - accuracy: 0.9937
148/375 [==========>...................] - ETA: 2s - loss: 0.0227 - accuracy: 0.9938
153/375 [===========>..................] - ETA: 2s - loss: 0.0226 - accuracy: 0.9938
159/375 [===========>..................] - ETA: 1s - loss: 0.0222 - accuracy: 0.9940
165/375 [============>.................] - ETA: 1s - loss: 0.0219 - accuracy: 0.9941
171/375 [============>.................] - ETA: 1s - loss: 0.0219 - accuracy: 0.9940
176/375 [=============>................] - ETA: 1s - loss: 0.0220 - accuracy: 0.9939
181/375 [=============>................] - ETA: 1s - loss: 0.0219 - accuracy: 0.9940
186/375 [=============>................] - ETA: 1s - loss: 0.0217 - accuracy: 0.9940
192/375 [==============>...............] - ETA: 1s - loss: 0.0217 - accuracy: 0.9941
198/375 [==============>...............] - ETA: 1s - loss: 0.0217 - accuracy: 0.9939
204/375 [===============>..............] - ETA: 1s - loss: 0.0215 - accuracy: 0.9940
210/375 [===============>..............] - ETA: 1s - loss: 0.0213 - accuracy: 0.9941
216/375 [================>.............] - ETA: 1s - loss: 0.0215 - accuracy: 0.9940
223/375 [================>.............] - ETA: 1s - loss: 0.0215 - accuracy: 0.9940
229/375 [=================>............] - ETA: 1s - loss: 0.0212 - accuracy: 0.9941
235/375 [=================>............] - ETA: 1s - loss: 0.0214 - accuracy: 0.9940
241/375 [==================>...........] - ETA: 1s - loss: 0.0212 - accuracy: 0.9940
247/375 [==================>...........] - ETA: 1s - loss: 0.0211 - accuracy: 0.9941
253/375 [===================>..........] - ETA: 1s - loss: 0.0214 - accuracy: 0.9939
258/375 [===================>..........] - ETA: 1s - loss: 0.0213 - accuracy: 0.9940
263/375 [====================>.........] - ETA: 1s - loss: 0.0212 - accuracy: 0.9940
269/375 [====================>.........] - ETA: 0s - loss: 0.0211 - accuracy: 0.9941
275/375 [=====================>........] - ETA: 0s - loss: 0.0210 - accuracy: 0.9941
281/375 [=====================>........] - ETA: 0s - loss: 0.0213 - accuracy: 0.9941
288/375 [======================>.......] - ETA: 0s - loss: 0.0214 - accuracy: 0.9941
293/375 [======================>.......] - ETA: 0s - loss: 0.0213 - accuracy: 0.9941
298/375 [======================>.......] - ETA: 0s - loss: 0.0212 - accuracy: 0.9942
304/375 [=======================>......] - ETA: 0s - loss: 0.0214 - accuracy: 0.9941
310/375 [=======================>......] - ETA: 0s - loss: 0.0217 - accuracy: 0.9940
315/375 [========================>.....] - ETA: 0s - loss: 0.0217 - accuracy: 0.9941
321/375 [========================>.....] - ETA: 0s - loss: 0.0218 - accuracy: 0.9941
327/375 [=========================>....] - ETA: 0s - loss: 0.0218 - accuracy: 0.9941
333/375 [=========================>....] - ETA: 0s - loss: 0.0219 - accuracy: 0.9940
339/375 [==========================>...] - ETA: 0s - loss: 0.0218 - accuracy: 0.9940
345/375 [==========================>...] - ETA: 0s - loss: 0.0218 - accuracy: 0.9941
347/375 [==========================>...] - ETA: 0s - loss: 0.0217 - accuracy: 0.9941
352/375 [===========================>..] - ETA: 0s - loss: 0.0217 - accuracy: 0.9941
356/375 [===========================>..] - ETA: 0s - loss: 0.0217 - accuracy: 0.9941
361/375 [===========================>..] - ETA: 0s - loss: 0.0217 - accuracy: 0.9940
366/375 [============================>.] - ETA: 0s - loss: 0.0217 - accuracy: 0.9941
371/375 [============================>.] - ETA: 0s - loss: 0.0216 - accuracy: 0.9941
375/375 [==============================] - 4s 10ms/step - loss: 0.0215 - accuracy: 0.9941

375/375 [==============================] - 5s 13ms/step - loss: 0.0215 - accuracy: 0.9941 - val_loss: 0.1079 - val_accuracy: 0.9724 - lr: 1.0000e-06
Epoch 19/20

  1/375 [..............................] - ETA: 3s - loss: 0.0028 - accuracy: 1.0000
  7/375 [..............................] - ETA: 3s - loss: 0.0182 - accuracy: 0.9955
 13/375 [>.............................] - ETA: 3s - loss: 0.0188 - accuracy: 0.9946
 20/375 [>.............................] - ETA: 2s - loss: 0.0195 - accuracy: 0.9938
 26/375 [=>............................] - ETA: 2s - loss: 0.0195 - accuracy: 0.9937
 32/375 [=>............................] - ETA: 2s - loss: 0.0202 - accuracy: 0.9937
 38/375 [==>...........................] - ETA: 2s - loss: 0.0208 - accuracy: 0.9934
 42/375 [==>...........................] - ETA: 3s - loss: 0.0200 - accuracy: 0.9940
 47/375 [==>...........................] - ETA: 3s - loss: 0.0200 - accuracy: 0.9940
 53/375 [===>..........................] - ETA: 2s - loss: 0.0206 - accuracy: 0.9937
 60/375 [===>..........................] - ETA: 2s - loss: 0.0210 - accuracy: 0.9934
 66/375 [====>.........................] - ETA: 2s - loss: 0.0217 - accuracy: 0.9935
 72/375 [====>.........................] - ETA: 2s - loss: 0.0226 - accuracy: 0.9935
 79/375 [=====>........................] - ETA: 2s - loss: 0.0228 - accuracy: 0.9935
 85/375 [=====>........................] - ETA: 2s - loss: 0.0220 - accuracy: 0.9938
 90/375 [======>.......................] - ETA: 2s - loss: 0.0214 - accuracy: 0.9941
 96/375 [======>.......................] - ETA: 2s - loss: 0.0211 - accuracy: 0.9941
101/375 [=======>......................] - ETA: 2s - loss: 0.0212 - accuracy: 0.9940
107/375 [=======>......................] - ETA: 2s - loss: 0.0214 - accuracy: 0.9939
114/375 [========>.....................] - ETA: 2s - loss: 0.0209 - accuracy: 0.9940
121/375 [========>.....................] - ETA: 2s - loss: 0.0206 - accuracy: 0.9941
128/375 [=========>....................] - ETA: 2s - loss: 0.0208 - accuracy: 0.9941
133/375 [=========>....................] - ETA: 2s - loss: 0.0205 - accuracy: 0.9942
139/375 [==========>...................] - ETA: 2s - loss: 0.0208 - accuracy: 0.9942
144/375 [==========>...................] - ETA: 2s - loss: 0.0209 - accuracy: 0.9941
149/375 [==========>...................] - ETA: 2s - loss: 0.0208 - accuracy: 0.9941
154/375 [===========>..................] - ETA: 2s - loss: 0.0210 - accuracy: 0.9941
159/375 [===========>..................] - ETA: 2s - loss: 0.0211 - accuracy: 0.9940
164/375 [============>.................] - ETA: 1s - loss: 0.0211 - accuracy: 0.9940
170/375 [============>.................] - ETA: 1s - loss: 0.0210 - accuracy: 0.9941
176/375 [=============>................] - ETA: 1s - loss: 0.0211 - accuracy: 0.9941
182/375 [=============>................] - ETA: 1s - loss: 0.0221 - accuracy: 0.9939
188/375 [==============>...............] - ETA: 1s - loss: 0.0219 - accuracy: 0.9940
193/375 [==============>...............] - ETA: 1s - loss: 0.0220 - accuracy: 0.9939
198/375 [==============>...............] - ETA: 1s - loss: 0.0221 - accuracy: 0.9938
204/375 [===============>..............] - ETA: 1s - loss: 0.0220 - accuracy: 0.9939
210/375 [===============>..............] - ETA: 1s - loss: 0.0217 - accuracy: 0.9940
216/375 [================>.............] - ETA: 1s - loss: 0.0218 - accuracy: 0.9939
222/375 [================>.............] - ETA: 1s - loss: 0.0217 - accuracy: 0.9939
226/375 [=================>............] - ETA: 1s - loss: 0.0224 - accuracy: 0.9937
229/375 [=================>............] - ETA: 1s - loss: 0.0224 - accuracy: 0.9935
235/375 [=================>............] - ETA: 1s - loss: 0.0222 - accuracy: 0.9937
241/375 [==================>...........] - ETA: 1s - loss: 0.0219 - accuracy: 0.9938
246/375 [==================>...........] - ETA: 1s - loss: 0.0217 - accuracy: 0.9938
252/375 [===================>..........] - ETA: 1s - loss: 0.0216 - accuracy: 0.9939
257/375 [===================>..........] - ETA: 1s - loss: 0.0214 - accuracy: 0.9940
263/375 [====================>.........] - ETA: 1s - loss: 0.0212 - accuracy: 0.9939
268/375 [====================>.........] - ETA: 1s - loss: 0.0213 - accuracy: 0.9940
274/375 [====================>.........] - ETA: 0s - loss: 0.0212 - accuracy: 0.9940
280/375 [=====================>........] - ETA: 0s - loss: 0.0212 - accuracy: 0.9940
286/375 [=====================>........] - ETA: 0s - loss: 0.0212 - accuracy: 0.9940
292/375 [======================>.......] - ETA: 0s - loss: 0.0214 - accuracy: 0.9940
298/375 [======================>.......] - ETA: 0s - loss: 0.0213 - accuracy: 0.9940
304/375 [=======================>......] - ETA: 0s - loss: 0.0212 - accuracy: 0.9941
310/375 [=======================>......] - ETA: 0s - loss: 0.0210 - accuracy: 0.9942
315/375 [========================>.....] - ETA: 0s - loss: 0.0210 - accuracy: 0.9942
321/375 [========================>.....] - ETA: 0s - loss: 0.0211 - accuracy: 0.9943
327/375 [=========================>....] - ETA: 0s - loss: 0.0211 - accuracy: 0.9943
331/375 [=========================>....] - ETA: 0s - loss: 0.0214 - accuracy: 0.9941
333/375 [=========================>....] - ETA: 0s - loss: 0.0213 - accuracy: 0.9942
337/375 [=========================>....] - ETA: 0s - loss: 0.0214 - accuracy: 0.9942
341/375 [==========================>...] - ETA: 0s - loss: 0.0215 - accuracy: 0.9942
345/375 [==========================>...] - ETA: 0s - loss: 0.0214 - accuracy: 0.9942
349/375 [==========================>...] - ETA: 0s - loss: 0.0214 - accuracy: 0.9942
355/375 [===========================>..] - ETA: 0s - loss: 0.0216 - accuracy: 0.9941
360/375 [===========================>..] - ETA: 0s - loss: 0.0216 - accuracy: 0.9941
364/375 [============================>.] - ETA: 0s - loss: 0.0216 - accuracy: 0.9941
370/375 [============================>.] - ETA: 0s - loss: 0.0215 - accuracy: 0.9941
375/375 [==============================] - 4s 10ms/step - loss: 0.0215 - accuracy: 0.9941

375/375 [==============================] - 5s 13ms/step - loss: 0.0215 - accuracy: 0.9941 - val_loss: 0.1079 - val_accuracy: 0.9724 - lr: 1.0000e-07
Epoch 20/20

  1/375 [..............................] - ETA: 2s - loss: 0.0226 - accuracy: 0.9922
  7/375 [..............................] - ETA: 3s - loss: 0.0145 - accuracy: 0.9967
 14/375 [>.............................] - ETA: 3s - loss: 0.0158 - accuracy: 0.9967
 20/375 [>.............................] - ETA: 2s - loss: 0.0140 - accuracy: 0.9969
 26/375 [=>............................] - ETA: 2s - loss: 0.0153 - accuracy: 0.9964
 32/375 [=>............................] - ETA: 2s - loss: 0.0171 - accuracy: 0.9958
 38/375 [==>...........................] - ETA: 2s - loss: 0.0186 - accuracy: 0.9949
 44/375 [==>...........................] - ETA: 2s - loss: 0.0176 - accuracy: 0.9952
 48/375 [==>...........................] - ETA: 2s - loss: 0.0177 - accuracy: 0.9951
 54/375 [===>..........................] - ETA: 2s - loss: 0.0175 - accuracy: 0.9952
 60/375 [===>..........................] - ETA: 2s - loss: 0.0181 - accuracy: 0.9949
 66/375 [====>.........................] - ETA: 2s - loss: 0.0182 - accuracy: 0.9950
 73/375 [====>.........................] - ETA: 2s - loss: 0.0188 - accuracy: 0.9950
 80/375 [=====>........................] - ETA: 2s - loss: 0.0184 - accuracy: 0.9952
 87/375 [=====>........................] - ETA: 2s - loss: 0.0180 - accuracy: 0.9952
 94/375 [======>.......................] - ETA: 2s - loss: 0.0181 - accuracy: 0.9953
101/375 [=======>......................] - ETA: 2s - loss: 0.0188 - accuracy: 0.9950
108/375 [=======>......................] - ETA: 2s - loss: 0.0184 - accuracy: 0.9952
115/375 [========>.....................] - ETA: 2s - loss: 0.0182 - accuracy: 0.9952
122/375 [========>.....................] - ETA: 2s - loss: 0.0186 - accuracy: 0.9952
129/375 [=========>....................] - ETA: 2s - loss: 0.0186 - accuracy: 0.9952
136/375 [=========>....................] - ETA: 1s - loss: 0.0187 - accuracy: 0.9950
143/375 [==========>...................] - ETA: 1s - loss: 0.0191 - accuracy: 0.9949
150/375 [===========>..................] - ETA: 1s - loss: 0.0189 - accuracy: 0.9950
156/375 [===========>..................] - ETA: 1s - loss: 0.0191 - accuracy: 0.9948
162/375 [===========>..................] - ETA: 1s - loss: 0.0190 - accuracy: 0.9948
169/375 [============>.................] - ETA: 1s - loss: 0.0191 - accuracy: 0.9947
175/375 [=============>................] - ETA: 1s - loss: 0.0195 - accuracy: 0.9947
182/375 [=============>................] - ETA: 1s - loss: 0.0198 - accuracy: 0.9946
189/375 [==============>...............] - ETA: 1s - loss: 0.0199 - accuracy: 0.9946
196/375 [==============>...............] - ETA: 1s - loss: 0.0199 - accuracy: 0.9945
203/375 [===============>..............] - ETA: 1s - loss: 0.0198 - accuracy: 0.9946
210/375 [===============>..............] - ETA: 1s - loss: 0.0206 - accuracy: 0.9943
217/375 [================>.............] - ETA: 1s - loss: 0.0209 - accuracy: 0.9943
223/375 [================>.............] - ETA: 1s - loss: 0.0211 - accuracy: 0.9944
228/375 [=================>............] - ETA: 1s - loss: 0.0211 - accuracy: 0.9944
234/375 [=================>............] - ETA: 1s - loss: 0.0211 - accuracy: 0.9944
240/375 [==================>...........] - ETA: 1s - loss: 0.0208 - accuracy: 0.9945
246/375 [==================>...........] - ETA: 1s - loss: 0.0210 - accuracy: 0.9944
252/375 [===================>..........] - ETA: 1s - loss: 0.0209 - accuracy: 0.9945
258/375 [===================>..........] - ETA: 0s - loss: 0.0210 - accuracy: 0.9944
265/375 [====================>.........] - ETA: 0s - loss: 0.0208 - accuracy: 0.9944
272/375 [====================>.........] - ETA: 0s - loss: 0.0207 - accuracy: 0.9944
279/375 [=====================>........] - ETA: 0s - loss: 0.0206 - accuracy: 0.9944
286/375 [=====================>........] - ETA: 0s - loss: 0.0206 - accuracy: 0.9944
293/375 [======================>.......] - ETA: 0s - loss: 0.0208 - accuracy: 0.9945
299/375 [======================>.......] - ETA: 0s - loss: 0.0209 - accuracy: 0.9944
305/375 [=======================>......] - ETA: 0s - loss: 0.0208 - accuracy: 0.9944
311/375 [=======================>......] - ETA: 0s - loss: 0.0209 - accuracy: 0.9944
317/375 [========================>.....] - ETA: 0s - loss: 0.0210 - accuracy: 0.9944
323/375 [========================>.....] - ETA: 0s - loss: 0.0209 - accuracy: 0.9944
329/375 [=========================>....] - ETA: 0s - loss: 0.0210 - accuracy: 0.9943
336/375 [=========================>....] - ETA: 0s - loss: 0.0210 - accuracy: 0.9944
343/375 [==========================>...] - ETA: 0s - loss: 0.0213 - accuracy: 0.9943
349/375 [==========================>...] - ETA: 0s - loss: 0.0212 - accuracy: 0.9943
355/375 [===========================>..] - ETA: 0s - loss: 0.0212 - accuracy: 0.9943
361/375 [===========================>..] - ETA: 0s - loss: 0.0214 - accuracy: 0.9942
367/375 [============================>.] - ETA: 0s - loss: 0.0214 - accuracy: 0.9941
373/375 [============================>.] - ETA: 0s - loss: 0.0214 - accuracy: 0.9941
375/375 [==============================] - 3s 8ms/step - loss: 0.0215 - accuracy: 0.9941

375/375 [==============================] - 4s 11ms/step - loss: 0.0215 - accuracy: 0.9941 - val_loss: 0.1079 - val_accuracy: 0.9724 - lr: 1.0000e-08

model_2_layer_v3 <- keras_model_sequential() %>%
  layer_dense(units = 128, activation = 'relu', input_shape = 784) %>%
  layer_dense(units = 64, activation = 'relu') %>%
  layer_dense(units = 32, activation = 'relu') %>%
  layer_dense(units = 10, activation = 'softmax')

summary(model_2_layer_v3)

Model: "sequential_9"
___________________________________________________________________________________________________________
 Layer (type)                                   Output Shape                              Param #          
===========================================================================================================
 dense_26 (Dense)                               (None, 128)                               100480           
 dense_25 (Dense)                               (None, 64)                                8256             
 dense_24 (Dense)                               (None, 32)                                2080             
 dense_23 (Dense)                               (None, 10)                                330              
===========================================================================================================
Total params: 111,146
Trainable params: 111,146
Non-trainable params: 0
___________________________________________________________________________________________________________

model_2_layer_v3 %>% compile(
  loss = "categorical_crossentropy",
  optimizer = optimizer_sgd(lr = 0.1, momentum = 0.9),
  metrics = "accuracy"
)

Warning: the `lr` argument has been renamed to `learning_rate`.

history <- model_2_layer_v3 %>% fit(
  train_images, train_labels,
  validation_split = 0.2,
  batch_size = 128,
  epochs = 20,
  callback = list(
    # callback_early_stopping(patience = 3, restore_best_weights = TRUE, min_delta = 0.0001),
    callback_reduce_lr_on_plateau(patience = 1, factor = 0.1)
    )
  )

Epoch 1/20

2022-08-04 15:46:50.734996: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:113] Plugin optimizer for device_type GPU is enabled.

  1/375 [..............................] - ETA: 1:41 - loss: 2.3541 - accuracy: 0.1406

WARNING:tensorflow:Callback method `on_train_batch_end` is slow compared to the batch time (batch time: 0.0064s vs `on_train_batch_end` time: 0.0081s). Check your callbacks.


  7/375 [..............................] - ETA: 3s - loss: 2.1703 - accuracy: 0.3203  
 13/375 [>.............................] - ETA: 3s - loss: 1.8203 - accuracy: 0.4513
 19/375 [>.............................] - ETA: 3s - loss: 1.5922 - accuracy: 0.5107
 25/375 [=>............................] - ETA: 3s - loss: 1.4067 - accuracy: 0.5634
 30/375 [=>............................] - ETA: 3s - loss: 1.3049 - accuracy: 0.5953
 36/375 [=>............................] - ETA: 3s - loss: 1.1928 - accuracy: 0.6287
 42/375 [==>...........................] - ETA: 2s - loss: 1.1077 - accuracy: 0.6533
 49/375 [==>...........................] - ETA: 2s - loss: 1.0267 - accuracy: 0.6790
 55/375 [===>..........................] - ETA: 2s - loss: 0.9697 - accuracy: 0.6976
 61/375 [===>..........................] - ETA: 2s - loss: 0.9222 - accuracy: 0.7112
 66/375 [====>.........................] - ETA: 2s - loss: 0.8944 - accuracy: 0.7209
 72/375 [====>.........................] - ETA: 2s - loss: 0.8468 - accuracy: 0.7354
 78/375 [=====>........................] - ETA: 2s - loss: 0.8120 - accuracy: 0.7467
 85/375 [=====>........................] - ETA: 2s - loss: 0.7767 - accuracy: 0.7579
 92/375 [======>.......................] - ETA: 2s - loss: 0.7432 - accuracy: 0.7690
 98/375 [======>.......................] - ETA: 2s - loss: 0.7155 - accuracy: 0.7779
104/375 [=======>......................] - ETA: 2s - loss: 0.6909 - accuracy: 0.7855
110/375 [=======>......................] - ETA: 2s - loss: 0.6694 - accuracy: 0.7920
116/375 [========>.....................] - ETA: 2s - loss: 0.6488 - accuracy: 0.7983
123/375 [========>.....................] - ETA: 2s - loss: 0.6293 - accuracy: 0.8042
129/375 [=========>....................] - ETA: 2s - loss: 0.6146 - accuracy: 0.8090
135/375 [=========>....................] - ETA: 2s - loss: 0.5986 - accuracy: 0.8144
141/375 [==========>...................] - ETA: 2s - loss: 0.5842 - accuracy: 0.8189
148/375 [==========>...................] - ETA: 1s - loss: 0.5706 - accuracy: 0.8236
155/375 [===========>..................] - ETA: 1s - loss: 0.5560 - accuracy: 0.8285
161/375 [===========>..................] - ETA: 1s - loss: 0.5426 - accuracy: 0.8326
168/375 [============>.................] - ETA: 1s - loss: 0.5301 - accuracy: 0.8362
170/375 [============>.................] - ETA: 1s - loss: 0.5268 - accuracy: 0.8375
172/375 [============>.................] - ETA: 1s - loss: 0.5227 - accuracy: 0.8387
178/375 [=============>................] - ETA: 1s - loss: 0.5144 - accuracy: 0.8415
184/375 [=============>................] - ETA: 1s - loss: 0.5038 - accuracy: 0.8449
190/375 [==============>...............] - ETA: 1s - loss: 0.4951 - accuracy: 0.8475
196/375 [==============>...............] - ETA: 1s - loss: 0.4858 - accuracy: 0.8507
203/375 [===============>..............] - ETA: 1s - loss: 0.4758 - accuracy: 0.8541
209/375 [===============>..............] - ETA: 1s - loss: 0.4688 - accuracy: 0.8564
216/375 [================>.............] - ETA: 1s - loss: 0.4598 - accuracy: 0.8590
223/375 [================>.............] - ETA: 1s - loss: 0.4517 - accuracy: 0.8614
230/375 [=================>............] - ETA: 1s - loss: 0.4450 - accuracy: 0.8634
237/375 [=================>............] - ETA: 1s - loss: 0.4396 - accuracy: 0.8651
244/375 [==================>...........] - ETA: 1s - loss: 0.4333 - accuracy: 0.8671
248/375 [==================>...........] - ETA: 1s - loss: 0.4296 - accuracy: 0.8684
255/375 [===================>..........] - ETA: 1s - loss: 0.4228 - accuracy: 0.8705
262/375 [===================>..........] - ETA: 0s - loss: 0.4169 - accuracy: 0.8721
269/375 [====================>.........] - ETA: 0s - loss: 0.4105 - accuracy: 0.8741
275/375 [=====================>........] - ETA: 0s - loss: 0.4048 - accuracy: 0.8757
281/375 [=====================>........] - ETA: 0s - loss: 0.4001 - accuracy: 0.8773
287/375 [=====================>........] - ETA: 0s - loss: 0.3957 - accuracy: 0.8788
293/375 [======================>.......] - ETA: 0s - loss: 0.3910 - accuracy: 0.8803
299/375 [======================>.......] - ETA: 0s - loss: 0.3873 - accuracy: 0.8814
305/375 [=======================>......] - ETA: 0s - loss: 0.3831 - accuracy: 0.8827
311/375 [=======================>......] - ETA: 0s - loss: 0.3788 - accuracy: 0.8840
317/375 [========================>.....] - ETA: 0s - loss: 0.3758 - accuracy: 0.8850
323/375 [========================>.....] - ETA: 0s - loss: 0.3727 - accuracy: 0.8859
325/375 [=========================>....] - ETA: 0s - loss: 0.3724 - accuracy: 0.8861
331/375 [=========================>....] - ETA: 0s - loss: 0.3703 - accuracy: 0.8868
337/375 [=========================>....] - ETA: 0s - loss: 0.3676 - accuracy: 0.8878
343/375 [==========================>...] - ETA: 0s - loss: 0.3644 - accuracy: 0.8888
350/375 [===========================>..] - ETA: 0s - loss: 0.3605 - accuracy: 0.8900
355/375 [===========================>..] - ETA: 0s - loss: 0.3583 - accuracy: 0.8906
361/375 [===========================>..] - ETA: 0s - loss: 0.3551 - accuracy: 0.8916
367/375 [============================>.] - ETA: 0s - loss: 0.3521 - accuracy: 0.8925
373/375 [============================>.] - ETA: 0s - loss: 0.3490 - accuracy: 0.8935
375/375 [==============================] - 4s 9ms/step - loss: 0.3481 - accuracy: 0.8939

2022-08-04 15:46:54.280252: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:113] Plugin optimizer for device_type GPU is enabled.


375/375 [==============================] - 5s 13ms/step - loss: 0.3481 - accuracy: 0.8939 - val_loss: 0.2019 - val_accuracy: 0.9408 - lr: 0.1000
Epoch 2/20

  1/375 [..............................] - ETA: 12s - loss: 0.1440 - accuracy: 0.9453
  7/375 [..............................] - ETA: 3s - loss: 0.1623 - accuracy: 0.9453 
 13/375 [>.............................] - ETA: 3s - loss: 0.1686 - accuracy: 0.9489
 19/375 [>.............................] - ETA: 3s - loss: 0.1571 - accuracy: 0.9507
 25/375 [=>............................] - ETA: 3s - loss: 0.1570 - accuracy: 0.9519
 31/375 [=>............................] - ETA: 3s - loss: 0.1523 - accuracy: 0.9529
 38/375 [==>...........................] - ETA: 3s - loss: 0.1516 - accuracy: 0.9525
 44/375 [==>...........................] - ETA: 3s - loss: 0.1572 - accuracy: 0.9503
 50/375 [===>..........................] - ETA: 2s - loss: 0.1557 - accuracy: 0.9503
 56/375 [===>..........................] - ETA: 2s - loss: 0.1536 - accuracy: 0.9516
 63/375 [====>.........................] - ETA: 2s - loss: 0.1502 - accuracy: 0.9529
 69/375 [====>.........................] - ETA: 2s - loss: 0.1526 - accuracy: 0.9527
 75/375 [=====>........................] - ETA: 2s - loss: 0.1504 - accuracy: 0.9533
 81/375 [=====>........................] - ETA: 2s - loss: 0.1504 - accuracy: 0.9532
 87/375 [=====>........................] - ETA: 2s - loss: 0.1504 - accuracy: 0.9529
 93/375 [======>.......................] - ETA: 2s - loss: 0.1532 - accuracy: 0.9520
 99/375 [======>.......................] - ETA: 2s - loss: 0.1519 - accuracy: 0.9526
105/375 [=======>......................] - ETA: 2s - loss: 0.1518 - accuracy: 0.9530
111/375 [=======>......................] - ETA: 2s - loss: 0.1527 - accuracy: 0.9531
118/375 [========>.....................] - ETA: 2s - loss: 0.1517 - accuracy: 0.9531
124/375 [========>.....................] - ETA: 2s - loss: 0.1502 - accuracy: 0.9536
130/375 [=========>....................] - ETA: 2s - loss: 0.1483 - accuracy: 0.9541
137/375 [=========>....................] - ETA: 2s - loss: 0.1470 - accuracy: 0.9545
143/375 [==========>...................] - ETA: 2s - loss: 0.1463 - accuracy: 0.9548
149/375 [==========>...................] - ETA: 1s - loss: 0.1462 - accuracy: 0.9551
156/375 [===========>..................] - ETA: 1s - loss: 0.1460 - accuracy: 0.9553
162/375 [===========>..................] - ETA: 1s - loss: 0.1460 - accuracy: 0.9552
167/375 [============>.................] - ETA: 1s - loss: 0.1455 - accuracy: 0.9555
173/375 [============>.................] - ETA: 1s - loss: 0.1449 - accuracy: 0.9558
179/375 [=============>................] - ETA: 1s - loss: 0.1437 - accuracy: 0.9564
186/375 [=============>................] - ETA: 1s - loss: 0.1454 - accuracy: 0.9561
193/375 [==============>...............] - ETA: 1s - loss: 0.1445 - accuracy: 0.9564
199/375 [==============>...............] - ETA: 1s - loss: 0.1439 - accuracy: 0.9563
205/375 [===============>..............] - ETA: 1s - loss: 0.1441 - accuracy: 0.9565
212/375 [===============>..............] - ETA: 1s - loss: 0.1430 - accuracy: 0.9568
219/375 [================>.............] - ETA: 1s - loss: 0.1422 - accuracy: 0.9569
226/375 [=================>............] - ETA: 1s - loss: 0.1415 - accuracy: 0.9571
232/375 [=================>............] - ETA: 1s - loss: 0.1402 - accuracy: 0.9574
238/375 [==================>...........] - ETA: 1s - loss: 0.1406 - accuracy: 0.9574
245/375 [==================>...........] - ETA: 1s - loss: 0.1393 - accuracy: 0.9577
252/375 [===================>..........] - ETA: 1s - loss: 0.1399 - accuracy: 0.9575
259/375 [===================>..........] - ETA: 1s - loss: 0.1399 - accuracy: 0.9575
266/375 [====================>.........] - ETA: 0s - loss: 0.1399 - accuracy: 0.9575
272/375 [====================>.........] - ETA: 0s - loss: 0.1391 - accuracy: 0.9579
277/375 [=====================>........] - ETA: 0s - loss: 0.1386 - accuracy: 0.9581
283/375 [=====================>........] - ETA: 0s - loss: 0.1385 - accuracy: 0.9581
289/375 [======================>.......] - ETA: 0s - loss: 0.1384 - accuracy: 0.9580
295/375 [======================>.......] - ETA: 0s - loss: 0.1376 - accuracy: 0.9582
301/375 [=======================>......] - ETA: 0s - loss: 0.1379 - accuracy: 0.9581
307/375 [=======================>......] - ETA: 0s - loss: 0.1375 - accuracy: 0.9582
313/375 [========================>.....] - ETA: 0s - loss: 0.1378 - accuracy: 0.9581
318/375 [========================>.....] - ETA: 0s - loss: 0.1377 - accuracy: 0.9581
323/375 [========================>.....] - ETA: 0s - loss: 0.1378 - accuracy: 0.9582
329/375 [=========================>....] - ETA: 0s - loss: 0.1375 - accuracy: 0.9583
335/375 [=========================>....] - ETA: 0s - loss: 0.1373 - accuracy: 0.9583
341/375 [==========================>...] - ETA: 0s - loss: 0.1366 - accuracy: 0.9586
347/375 [==========================>...] - ETA: 0s - loss: 0.1365 - accuracy: 0.9586
352/375 [===========================>..] - ETA: 0s - loss: 0.1365 - accuracy: 0.9586
358/375 [===========================>..] - ETA: 0s - loss: 0.1363 - accuracy: 0.9586
363/375 [============================>.] - ETA: 0s - loss: 0.1364 - accuracy: 0.9587
369/375 [============================>.] - ETA: 0s - loss: 0.1366 - accuracy: 0.9586
374/375 [============================>.] - ETA: 0s - loss: 0.1366 - accuracy: 0.9586
375/375 [==============================] - 3s 9ms/step - loss: 0.1365 - accuracy: 0.9586

375/375 [==============================] - 4s 11ms/step - loss: 0.1365 - accuracy: 0.9586 - val_loss: 0.1550 - val_accuracy: 0.9532 - lr: 0.1000
Epoch 3/20

  1/375 [..............................] - ETA: 2s - loss: 0.2197 - accuracy: 0.9375
  7/375 [..............................] - ETA: 3s - loss: 0.1078 - accuracy: 0.9643
 14/375 [>.............................] - ETA: 3s - loss: 0.1054 - accuracy: 0.9660
 20/375 [>.............................] - ETA: 2s - loss: 0.1011 - accuracy: 0.9672
 26/375 [=>............................] - ETA: 2s - loss: 0.1028 - accuracy: 0.9675
 33/375 [=>............................] - ETA: 2s - loss: 0.1031 - accuracy: 0.9666
 39/375 [==>...........................] - ETA: 2s - loss: 0.1020 - accuracy: 0.9667
 45/375 [==>...........................] - ETA: 2s - loss: 0.1042 - accuracy: 0.9670
 50/375 [===>..........................] - ETA: 2s - loss: 0.1008 - accuracy: 0.9680
 55/375 [===>..........................] - ETA: 2s - loss: 0.1019 - accuracy: 0.9676
 60/375 [===>..........................] - ETA: 2s - loss: 0.1047 - accuracy: 0.9671
 66/375 [====>.........................] - ETA: 2s - loss: 0.1056 - accuracy: 0.9670
 72/375 [====>.........................] - ETA: 2s - loss: 0.1057 - accuracy: 0.9671
 78/375 [=====>........................] - ETA: 2s - loss: 0.1069 - accuracy: 0.9663
 84/375 [=====>........................] - ETA: 2s - loss: 0.1068 - accuracy: 0.9664
 90/375 [======>.......................] - ETA: 2s - loss: 0.1046 - accuracy: 0.9669
 96/375 [======>.......................] - ETA: 2s - loss: 0.1070 - accuracy: 0.9666
102/375 [=======>......................] - ETA: 2s - loss: 0.1068 - accuracy: 0.9671
108/375 [=======>......................] - ETA: 2s - loss: 0.1054 - accuracy: 0.9676
114/375 [========>.....................] - ETA: 2s - loss: 0.1077 - accuracy: 0.9674
120/375 [========>.....................] - ETA: 2s - loss: 0.1090 - accuracy: 0.9673
126/375 [=========>....................] - ETA: 2s - loss: 0.1081 - accuracy: 0.9675
132/375 [=========>....................] - ETA: 2s - loss: 0.1095 - accuracy: 0.9673
138/375 [==========>...................] - ETA: 2s - loss: 0.1105 - accuracy: 0.9669
144/375 [==========>...................] - ETA: 2s - loss: 0.1105 - accuracy: 0.9667
151/375 [===========>..................] - ETA: 2s - loss: 0.1111 - accuracy: 0.9667
157/375 [===========>..................] - ETA: 1s - loss: 0.1096 - accuracy: 0.9672
163/375 [============>.................] - ETA: 1s - loss: 0.1079 - accuracy: 0.9675
168/375 [============>.................] - ETA: 1s - loss: 0.1072 - accuracy: 0.9677
175/375 [=============>................] - ETA: 1s - loss: 0.1065 - accuracy: 0.9682
182/375 [=============>................] - ETA: 1s - loss: 0.1070 - accuracy: 0.9682
188/375 [==============>...............] - ETA: 1s - loss: 0.1064 - accuracy: 0.9685
194/375 [==============>...............] - ETA: 1s - loss: 0.1064 - accuracy: 0.9685
200/375 [===============>..............] - ETA: 1s - loss: 0.1059 - accuracy: 0.9687
207/375 [===============>..............] - ETA: 1s - loss: 0.1059 - accuracy: 0.9688
213/375 [================>.............] - ETA: 1s - loss: 0.1057 - accuracy: 0.9689
220/375 [================>.............] - ETA: 1s - loss: 0.1059 - accuracy: 0.9689
226/375 [=================>............] - ETA: 1s - loss: 0.1056 - accuracy: 0.9691
233/375 [=================>............] - ETA: 1s - loss: 0.1051 - accuracy: 0.9691
240/375 [==================>...........] - ETA: 1s - loss: 0.1044 - accuracy: 0.9692
246/375 [==================>...........] - ETA: 1s - loss: 0.1042 - accuracy: 0.9692
253/375 [===================>..........] - ETA: 1s - loss: 0.1032 - accuracy: 0.9694
259/375 [===================>..........] - ETA: 1s - loss: 0.1036 - accuracy: 0.9691
266/375 [====================>.........] - ETA: 0s - loss: 0.1030 - accuracy: 0.9691
272/375 [====================>.........] - ETA: 0s - loss: 0.1037 - accuracy: 0.9688
278/375 [=====================>........] - ETA: 0s - loss: 0.1035 - accuracy: 0.9688
285/375 [=====================>........] - ETA: 0s - loss: 0.1034 - accuracy: 0.9688
291/375 [======================>.......] - ETA: 0s - loss: 0.1025 - accuracy: 0.9690
297/375 [======================>.......] - ETA: 0s - loss: 0.1018 - accuracy: 0.9693
303/375 [=======================>......] - ETA: 0s - loss: 0.1009 - accuracy: 0.9695
309/375 [=======================>......] - ETA: 0s - loss: 0.1007 - accuracy: 0.9696
315/375 [========================>.....] - ETA: 0s - loss: 0.1001 - accuracy: 0.9697
321/375 [========================>.....] - ETA: 0s - loss: 0.1001 - accuracy: 0.9697
327/375 [=========================>....] - ETA: 0s - loss: 0.1006 - accuracy: 0.9696
333/375 [=========================>....] - ETA: 0s - loss: 0.1000 - accuracy: 0.9698
339/375 [==========================>...] - ETA: 0s - loss: 0.0995 - accuracy: 0.9699
346/375 [==========================>...] - ETA: 0s - loss: 0.0996 - accuracy: 0.9699
353/375 [===========================>..] - ETA: 0s - loss: 0.0989 - accuracy: 0.9701
359/375 [===========================>..] - ETA: 0s - loss: 0.0986 - accuracy: 0.9703
365/375 [============================>.] - ETA: 0s - loss: 0.0982 - accuracy: 0.9704
371/375 [============================>.] - ETA: 0s - loss: 0.0988 - accuracy: 0.9702
375/375 [==============================] - 3s 9ms/step - loss: 0.0988 - accuracy: 0.9701

375/375 [==============================] - 4s 11ms/step - loss: 0.0988 - accuracy: 0.9701 - val_loss: 0.1255 - val_accuracy: 0.9661 - lr: 0.1000
Epoch 4/20

  1/375 [..............................] - ETA: 3s - loss: 0.0964 - accuracy: 0.9844
  7/375 [..............................] - ETA: 3s - loss: 0.0660 - accuracy: 0.9777
 13/375 [>.............................] - ETA: 3s - loss: 0.0734 - accuracy: 0.9784
 20/375 [>.............................] - ETA: 3s - loss: 0.0848 - accuracy: 0.9734
 27/375 [=>............................] - ETA: 2s - loss: 0.0847 - accuracy: 0.9751
 33/375 [=>............................] - ETA: 2s - loss: 0.0805 - accuracy: 0.9759
 40/375 [==>...........................] - ETA: 2s - loss: 0.0813 - accuracy: 0.9768
 46/375 [==>...........................] - ETA: 2s - loss: 0.0781 - accuracy: 0.9776
 52/375 [===>..........................] - ETA: 2s - loss: 0.0759 - accuracy: 0.9779
 58/375 [===>..........................] - ETA: 2s - loss: 0.0737 - accuracy: 0.9787
 64/375 [====>.........................] - ETA: 2s - loss: 0.0751 - accuracy: 0.9783
 70/375 [====>.........................] - ETA: 2s - loss: 0.0742 - accuracy: 0.9782
 76/375 [=====>........................] - ETA: 2s - loss: 0.0720 - accuracy: 0.9785
 83/375 [=====>........................] - ETA: 2s - loss: 0.0756 - accuracy: 0.9774
 89/375 [======>.......................] - ETA: 2s - loss: 0.0746 - accuracy: 0.9779
 95/375 [======>.......................] - ETA: 2s - loss: 0.0737 - accuracy: 0.9780
101/375 [=======>......................] - ETA: 2s - loss: 0.0728 - accuracy: 0.9780
107/375 [=======>......................] - ETA: 2s - loss: 0.0736 - accuracy: 0.9777
111/375 [=======>......................] - ETA: 2s - loss: 0.0743 - accuracy: 0.9773
117/375 [========>.....................] - ETA: 2s - loss: 0.0754 - accuracy: 0.9770
123/375 [========>.....................] - ETA: 2s - loss: 0.0735 - accuracy: 0.9777
130/375 [=========>....................] - ETA: 2s - loss: 0.0735 - accuracy: 0.9772
137/375 [=========>....................] - ETA: 2s - loss: 0.0748 - accuracy: 0.9767
143/375 [==========>...................] - ETA: 2s - loss: 0.0755 - accuracy: 0.9765
150/375 [===========>..................] - ETA: 1s - loss: 0.0750 - accuracy: 0.9766
156/375 [===========>..................] - ETA: 1s - loss: 0.0751 - accuracy: 0.9768
163/375 [============>.................] - ETA: 1s - loss: 0.0774 - accuracy: 0.9760
169/375 [============>.................] - ETA: 1s - loss: 0.0788 - accuracy: 0.9758
175/375 [=============>................] - ETA: 1s - loss: 0.0793 - accuracy: 0.9757
181/375 [=============>................] - ETA: 1s - loss: 0.0794 - accuracy: 0.9759
187/375 [=============>................] - ETA: 1s - loss: 0.0788 - accuracy: 0.9761
193/375 [==============>...............] - ETA: 1s - loss: 0.0790 - accuracy: 0.9761
200/375 [===============>..............] - ETA: 1s - loss: 0.0793 - accuracy: 0.9759
206/375 [===============>..............] - ETA: 1s - loss: 0.0799 - accuracy: 0.9755
212/375 [===============>..............] - ETA: 1s - loss: 0.0806 - accuracy: 0.9753
218/375 [================>.............] - ETA: 1s - loss: 0.0811 - accuracy: 0.9752
224/375 [================>.............] - ETA: 1s - loss: 0.0818 - accuracy: 0.9750
229/375 [=================>............] - ETA: 1s - loss: 0.0817 - accuracy: 0.9751
235/375 [=================>............] - ETA: 1s - loss: 0.0821 - accuracy: 0.9751
241/375 [==================>...........] - ETA: 1s - loss: 0.0819 - accuracy: 0.9751
246/375 [==================>...........] - ETA: 1s - loss: 0.0822 - accuracy: 0.9749
252/375 [===================>..........] - ETA: 1s - loss: 0.0816 - accuracy: 0.9752
258/375 [===================>..........] - ETA: 1s - loss: 0.0813 - accuracy: 0.9752
264/375 [====================>.........] - ETA: 1s - loss: 0.0812 - accuracy: 0.9752
270/375 [====================>.........] - ETA: 0s - loss: 0.0812 - accuracy: 0.9752
276/375 [=====================>........] - ETA: 0s - loss: 0.0813 - accuracy: 0.9752
281/375 [=====================>........] - ETA: 0s - loss: 0.0813 - accuracy: 0.9752
287/375 [=====================>........] - ETA: 0s - loss: 0.0812 - accuracy: 0.9752
293/375 [======================>.......] - ETA: 0s - loss: 0.0809 - accuracy: 0.9753
298/375 [======================>.......] - ETA: 0s - loss: 0.0808 - accuracy: 0.9754
304/375 [=======================>......] - ETA: 0s - loss: 0.0806 - accuracy: 0.9754
310/375 [=======================>......] - ETA: 0s - loss: 0.0805 - accuracy: 0.9754
316/375 [========================>.....] - ETA: 0s - loss: 0.0808 - accuracy: 0.9753
322/375 [========================>.....] - ETA: 0s - loss: 0.0806 - accuracy: 0.9753
327/375 [=========================>....] - ETA: 0s - loss: 0.0798 - accuracy: 0.9756
332/375 [=========================>....] - ETA: 0s - loss: 0.0795 - accuracy: 0.9757
338/375 [==========================>...] - ETA: 0s - loss: 0.0797 - accuracy: 0.9756
343/375 [==========================>...] - ETA: 0s - loss: 0.0798 - accuracy: 0.9755
349/375 [==========================>...] - ETA: 0s - loss: 0.0801 - accuracy: 0.9754
355/375 [===========================>..] - ETA: 0s - loss: 0.0807 - accuracy: 0.9752
360/375 [===========================>..] - ETA: 0s - loss: 0.0806 - accuracy: 0.9752
366/375 [============================>.] - ETA: 0s - loss: 0.0806 - accuracy: 0.9752
371/375 [============================>.] - ETA: 0s - loss: 0.0809 - accuracy: 0.9752
375/375 [==============================] - 4s 9ms/step - loss: 0.0810 - accuracy: 0.9751

375/375 [==============================] - 5s 12ms/step - loss: 0.0810 - accuracy: 0.9751 - val_loss: 0.1003 - val_accuracy: 0.9708 - lr: 0.1000
Epoch 5/20

  1/375 [..............................] - ETA: 4s - loss: 0.0415 - accuracy: 0.9922
  8/375 [..............................] - ETA: 3s - loss: 0.0374 - accuracy: 0.9883
 14/375 [>.............................] - ETA: 3s - loss: 0.0366 - accuracy: 0.9894
 20/375 [>.............................] - ETA: 3s - loss: 0.0369 - accuracy: 0.9902
 27/375 [=>............................] - ETA: 2s - loss: 0.0398 - accuracy: 0.9890
 33/375 [=>............................] - ETA: 2s - loss: 0.0419 - accuracy: 0.9889
 39/375 [==>...........................] - ETA: 2s - loss: 0.0453 - accuracy: 0.9870
 45/375 [==>...........................] - ETA: 2s - loss: 0.0477 - accuracy: 0.9854
 51/375 [===>..........................] - ETA: 2s - loss: 0.0473 - accuracy: 0.9851
 56/375 [===>..........................] - ETA: 2s - loss: 0.0470 - accuracy: 0.9854
 59/375 [===>..........................] - ETA: 2s - loss: 0.0478 - accuracy: 0.9853
 65/375 [====>.........................] - ETA: 2s - loss: 0.0474 - accuracy: 0.9850
 70/375 [====>.........................] - ETA: 2s - loss: 0.0469 - accuracy: 0.9849
 76/375 [=====>........................] - ETA: 2s - loss: 0.0488 - accuracy: 0.9844
 82/375 [=====>........................] - ETA: 2s - loss: 0.0478 - accuracy: 0.9846
 88/375 [======>.......................] - ETA: 2s - loss: 0.0500 - accuracy: 0.9841
 94/375 [======>.......................] - ETA: 2s - loss: 0.0517 - accuracy: 0.9835
 99/375 [======>.......................] - ETA: 2s - loss: 0.0519 - accuracy: 0.9834
105/375 [=======>......................] - ETA: 2s - loss: 0.0541 - accuracy: 0.9830
111/375 [=======>......................] - ETA: 2s - loss: 0.0556 - accuracy: 0.9828
116/375 [========>.....................] - ETA: 2s - loss: 0.0562 - accuracy: 0.9825
121/375 [========>.....................] - ETA: 2s - loss: 0.0565 - accuracy: 0.9824
127/375 [=========>....................] - ETA: 2s - loss: 0.0570 - accuracy: 0.9820
132/375 [=========>....................] - ETA: 2s - loss: 0.0574 - accuracy: 0.9818
138/375 [==========>...................] - ETA: 2s - loss: 0.0578 - accuracy: 0.9819
145/375 [==========>...................] - ETA: 2s - loss: 0.0593 - accuracy: 0.9810
151/375 [===========>..................] - ETA: 2s - loss: 0.0619 - accuracy: 0.9804
157/375 [===========>..................] - ETA: 2s - loss: 0.0619 - accuracy: 0.9804
163/375 [============>.................] - ETA: 2s - loss: 0.0623 - accuracy: 0.9805
169/375 [============>.................] - ETA: 2s - loss: 0.0618 - accuracy: 0.9807
174/375 [============>.................] - ETA: 1s - loss: 0.0623 - accuracy: 0.9805
180/375 [=============>................] - ETA: 1s - loss: 0.0620 - accuracy: 0.9805
185/375 [=============>................] - ETA: 1s - loss: 0.0615 - accuracy: 0.9806
190/375 [==============>...............] - ETA: 1s - loss: 0.0610 - accuracy: 0.9808
195/375 [==============>...............] - ETA: 1s - loss: 0.0609 - accuracy: 0.9807
200/375 [===============>..............] - ETA: 1s - loss: 0.0608 - accuracy: 0.9807
206/375 [===============>..............] - ETA: 1s - loss: 0.0607 - accuracy: 0.9808
212/375 [===============>..............] - ETA: 1s - loss: 0.0601 - accuracy: 0.9810
217/375 [================>.............] - ETA: 1s - loss: 0.0598 - accuracy: 0.9811
223/375 [================>.............] - ETA: 1s - loss: 0.0602 - accuracy: 0.9810
228/375 [=================>............] - ETA: 1s - loss: 0.0608 - accuracy: 0.9808
233/375 [=================>............] - ETA: 1s - loss: 0.0609 - accuracy: 0.9808
236/375 [=================>............] - ETA: 1s - loss: 0.0610 - accuracy: 0.9807
241/375 [==================>...........] - ETA: 1s - loss: 0.0607 - accuracy: 0.9808
247/375 [==================>...........] - ETA: 1s - loss: 0.0612 - accuracy: 0.9807
253/375 [===================>..........] - ETA: 1s - loss: 0.0612 - accuracy: 0.9806
258/375 [===================>..........] - ETA: 1s - loss: 0.0617 - accuracy: 0.9806
263/375 [====================>.........] - ETA: 1s - loss: 0.0615 - accuracy: 0.9806
269/375 [====================>.........] - ETA: 1s - loss: 0.0615 - accuracy: 0.9805
274/375 [====================>.........] - ETA: 1s - loss: 0.0615 - accuracy: 0.9804
280/375 [=====================>........] - ETA: 0s - loss: 0.0618 - accuracy: 0.9804
285/375 [=====================>........] - ETA: 0s - loss: 0.0621 - accuracy: 0.9804
291/375 [======================>.......] - ETA: 0s - loss: 0.0623 - accuracy: 0.9803
297/375 [======================>.......] - ETA: 0s - loss: 0.0628 - accuracy: 0.9803
302/375 [=======================>......] - ETA: 0s - loss: 0.0626 - accuracy: 0.9803
307/375 [=======================>......] - ETA: 0s - loss: 0.0625 - accuracy: 0.9804
313/375 [========================>.....] - ETA: 0s - loss: 0.0625 - accuracy: 0.9804
318/375 [========================>.....] - ETA: 0s - loss: 0.0625 - accuracy: 0.9803
323/375 [========================>.....] - ETA: 0s - loss: 0.0628 - accuracy: 0.9803
329/375 [=========================>....] - ETA: 0s - loss: 0.0624 - accuracy: 0.9803
335/375 [=========================>....] - ETA: 0s - loss: 0.0621 - accuracy: 0.9804
341/375 [==========================>...] - ETA: 0s - loss: 0.0619 - accuracy: 0.9804
347/375 [==========================>...] - ETA: 0s - loss: 0.0619 - accuracy: 0.9804
353/375 [===========================>..] - ETA: 0s - loss: 0.0623 - accuracy: 0.9804
358/375 [===========================>..] - ETA: 0s - loss: 0.0628 - accuracy: 0.9803
363/375 [============================>.] - ETA: 0s - loss: 0.0631 - accuracy: 0.9803
369/375 [============================>.] - ETA: 0s - loss: 0.0634 - accuracy: 0.9801
375/375 [==============================] - 4s 10ms/step - loss: 0.0637 - accuracy: 0.9800

375/375 [==============================] - 5s 13ms/step - loss: 0.0637 - accuracy: 0.9800 - val_loss: 0.1121 - val_accuracy: 0.9688 - lr: 0.1000
Epoch 6/20

  1/375 [..............................] - ETA: 5s - loss: 0.0257 - accuracy: 1.0000
  6/375 [..............................] - ETA: 4s - loss: 0.0792 - accuracy: 0.9779
 12/375 [..............................] - ETA: 3s - loss: 0.0954 - accuracy: 0.9707
 18/375 [>.............................] - ETA: 3s - loss: 0.0890 - accuracy: 0.9731
 25/375 [=>............................] - ETA: 3s - loss: 0.0805 - accuracy: 0.9753
 32/375 [=>............................] - ETA: 3s - loss: 0.0841 - accuracy: 0.9739
 38/375 [==>...........................] - ETA: 3s - loss: 0.0787 - accuracy: 0.9757
 44/375 [==>...........................] - ETA: 3s - loss: 0.0735 - accuracy: 0.9771
 50/375 [===>..........................] - ETA: 3s - loss: 0.0691 - accuracy: 0.9786
 56/375 [===>..........................] - ETA: 2s - loss: 0.0663 - accuracy: 0.9792
 62/375 [===>..........................] - ETA: 2s - loss: 0.0633 - accuracy: 0.9805
 69/375 [====>.........................] - ETA: 2s - loss: 0.0601 - accuracy: 0.9813
 76/375 [=====>........................] - ETA: 2s - loss: 0.0576 - accuracy: 0.9822
 83/375 [=====>........................] - ETA: 2s - loss: 0.0568 - accuracy: 0.9825
 90/375 [======>.......................] - ETA: 2s - loss: 0.0555 - accuracy: 0.9827
 96/375 [======>.......................] - ETA: 2s - loss: 0.0540 - accuracy: 0.9832
103/375 [=======>......................] - ETA: 2s - loss: 0.0521 - accuracy: 0.9838
109/375 [=======>......................] - ETA: 2s - loss: 0.0517 - accuracy: 0.9839
116/375 [========>.....................] - ETA: 2s - loss: 0.0511 - accuracy: 0.9840
123/375 [========>.....................] - ETA: 2s - loss: 0.0496 - accuracy: 0.9844
130/375 [=========>....................] - ETA: 2s - loss: 0.0484 - accuracy: 0.9849
137/375 [=========>....................] - ETA: 2s - loss: 0.0474 - accuracy: 0.9853
144/375 [==========>...................] - ETA: 1s - loss: 0.0466 - accuracy: 0.9856
150/375 [===========>..................] - ETA: 1s - loss: 0.0463 - accuracy: 0.9856
156/375 [===========>..................] - ETA: 1s - loss: 0.0454 - accuracy: 0.9858
161/375 [===========>..................] - ETA: 1s - loss: 0.0451 - accuracy: 0.9859
167/375 [============>.................] - ETA: 1s - loss: 0.0447 - accuracy: 0.9861
172/375 [============>.................] - ETA: 1s - loss: 0.0441 - accuracy: 0.9863
178/375 [=============>................] - ETA: 1s - loss: 0.0439 - accuracy: 0.9865
184/375 [=============>................] - ETA: 1s - loss: 0.0432 - accuracy: 0.9868
190/375 [==============>...............] - ETA: 1s - loss: 0.0425 - accuracy: 0.9870
196/375 [==============>...............] - ETA: 1s - loss: 0.0423 - accuracy: 0.9872
202/375 [===============>..............] - ETA: 1s - loss: 0.0424 - accuracy: 0.9872
205/375 [===============>..............] - ETA: 1s - loss: 0.0422 - accuracy: 0.9872
208/375 [===============>..............] - ETA: 1s - loss: 0.0418 - accuracy: 0.9873
210/375 [===============>..............] - ETA: 1s - loss: 0.0418 - accuracy: 0.9874
216/375 [================>.............] - ETA: 1s - loss: 0.0415 - accuracy: 0.9873
222/375 [================>.............] - ETA: 1s - loss: 0.0416 - accuracy: 0.9874
228/375 [=================>............] - ETA: 1s - loss: 0.0416 - accuracy: 0.9874
235/375 [=================>............] - ETA: 1s - loss: 0.0412 - accuracy: 0.9875
241/375 [==================>...........] - ETA: 1s - loss: 0.0407 - accuracy: 0.9877
248/375 [==================>...........] - ETA: 1s - loss: 0.0409 - accuracy: 0.9876
255/375 [===================>..........] - ETA: 1s - loss: 0.0408 - accuracy: 0.9876
262/375 [===================>..........] - ETA: 1s - loss: 0.0403 - accuracy: 0.9877
269/375 [====================>.........] - ETA: 0s - loss: 0.0401 - accuracy: 0.9878
275/375 [=====================>........] - ETA: 0s - loss: 0.0397 - accuracy: 0.9879
281/375 [=====================>........] - ETA: 0s - loss: 0.0394 - accuracy: 0.9880
286/375 [=====================>........] - ETA: 0s - loss: 0.0389 - accuracy: 0.9881
292/375 [======================>.......] - ETA: 0s - loss: 0.0390 - accuracy: 0.9881
298/375 [======================>.......] - ETA: 0s - loss: 0.0387 - accuracy: 0.9882
305/375 [=======================>......] - ETA: 0s - loss: 0.0384 - accuracy: 0.9883
312/375 [=======================>......] - ETA: 0s - loss: 0.0381 - accuracy: 0.9884
318/375 [========================>.....] - ETA: 0s - loss: 0.0377 - accuracy: 0.9885
325/375 [=========================>....] - ETA: 0s - loss: 0.0374 - accuracy: 0.9886
332/375 [=========================>....] - ETA: 0s - loss: 0.0374 - accuracy: 0.9885
338/375 [==========================>...] - ETA: 0s - loss: 0.0370 - accuracy: 0.9887
345/375 [==========================>...] - ETA: 0s - loss: 0.0366 - accuracy: 0.9888
352/375 [===========================>..] - ETA: 0s - loss: 0.0363 - accuracy: 0.9889
359/375 [===========================>..] - ETA: 0s - loss: 0.0364 - accuracy: 0.9889
366/375 [============================>.] - ETA: 0s - loss: 0.0367 - accuracy: 0.9888
373/375 [============================>.] - ETA: 0s - loss: 0.0364 - accuracy: 0.9888
375/375 [==============================] - 3s 9ms/step - loss: 0.0365 - accuracy: 0.9888

375/375 [==============================] - 4s 11ms/step - loss: 0.0365 - accuracy: 0.9888 - val_loss: 0.0833 - val_accuracy: 0.9778 - lr: 0.0100
Epoch 7/20

  1/375 [..............................] - ETA: 5s - loss: 0.0186 - accuracy: 1.0000
  7/375 [..............................] - ETA: 3s - loss: 0.0202 - accuracy: 0.9955
 13/375 [>.............................] - ETA: 3s - loss: 0.0240 - accuracy: 0.9934
 19/375 [>.............................] - ETA: 3s - loss: 0.0221 - accuracy: 0.9942
 25/375 [=>............................] - ETA: 3s - loss: 0.0239 - accuracy: 0.9944
 31/375 [=>............................] - ETA: 3s - loss: 0.0233 - accuracy: 0.9945
 37/375 [=>............................] - ETA: 2s - loss: 0.0242 - accuracy: 0.9939
 42/375 [==>...........................] - ETA: 3s - loss: 0.0229 - accuracy: 0.9942
 48/375 [==>...........................] - ETA: 3s - loss: 0.0220 - accuracy: 0.9945
 55/375 [===>..........................] - ETA: 2s - loss: 0.0213 - accuracy: 0.9947
 62/375 [===>..........................] - ETA: 2s - loss: 0.0208 - accuracy: 0.9948
 69/375 [====>.........................] - ETA: 2s - loss: 0.0212 - accuracy: 0.9946
 76/375 [=====>........................] - ETA: 2s - loss: 0.0212 - accuracy: 0.9943
 83/375 [=====>........................] - ETA: 2s - loss: 0.0203 - accuracy: 0.9946
 90/375 [======>.......................] - ETA: 2s - loss: 0.0205 - accuracy: 0.9944
 97/375 [======>.......................] - ETA: 2s - loss: 0.0201 - accuracy: 0.9947
104/375 [=======>......................] - ETA: 2s - loss: 0.0205 - accuracy: 0.9947
111/375 [=======>......................] - ETA: 2s - loss: 0.0209 - accuracy: 0.9946
118/375 [========>.....................] - ETA: 2s - loss: 0.0208 - accuracy: 0.9945
125/375 [=========>....................] - ETA: 2s - loss: 0.0204 - accuracy: 0.9946
130/375 [=========>....................] - ETA: 2s - loss: 0.0205 - accuracy: 0.9945
137/375 [=========>....................] - ETA: 2s - loss: 0.0203 - accuracy: 0.9945
144/375 [==========>...................] - ETA: 1s - loss: 0.0205 - accuracy: 0.9945
151/375 [===========>..................] - ETA: 1s - loss: 0.0204 - accuracy: 0.9945
158/375 [===========>..................] - ETA: 1s - loss: 0.0207 - accuracy: 0.9943
165/375 [============>.................] - ETA: 1s - loss: 0.0208 - accuracy: 0.9944
172/375 [============>.................] - ETA: 1s - loss: 0.0205 - accuracy: 0.9945
179/375 [=============>................] - ETA: 1s - loss: 0.0205 - accuracy: 0.9945
186/375 [=============>................] - ETA: 1s - loss: 0.0202 - accuracy: 0.9947
193/375 [==============>...............] - ETA: 1s - loss: 0.0200 - accuracy: 0.9948
200/375 [===============>..............] - ETA: 1s - loss: 0.0203 - accuracy: 0.9946
207/375 [===============>..............] - ETA: 1s - loss: 0.0203 - accuracy: 0.9945
214/375 [================>.............] - ETA: 1s - loss: 0.0205 - accuracy: 0.9944
221/375 [================>.............] - ETA: 1s - loss: 0.0205 - accuracy: 0.9944
228/375 [=================>............] - ETA: 1s - loss: 0.0207 - accuracy: 0.9943
235/375 [=================>............] - ETA: 1s - loss: 0.0207 - accuracy: 0.9942
242/375 [==================>...........] - ETA: 1s - loss: 0.0208 - accuracy: 0.9942
249/375 [==================>...........] - ETA: 1s - loss: 0.0207 - accuracy: 0.9943
256/375 [===================>..........] - ETA: 0s - loss: 0.0206 - accuracy: 0.9943
262/375 [===================>..........] - ETA: 0s - loss: 0.0206 - accuracy: 0.9943
269/375 [====================>.........] - ETA: 0s - loss: 0.0206 - accuracy: 0.9942
276/375 [=====================>........] - ETA: 0s - loss: 0.0206 - accuracy: 0.9941
283/375 [=====================>........] - ETA: 0s - loss: 0.0205 - accuracy: 0.9941
290/375 [======================>.......] - ETA: 0s - loss: 0.0206 - accuracy: 0.9942
297/375 [======================>.......] - ETA: 0s - loss: 0.0209 - accuracy: 0.9940
303/375 [=======================>......] - ETA: 0s - loss: 0.0207 - accuracy: 0.9940
309/375 [=======================>......] - ETA: 0s - loss: 0.0208 - accuracy: 0.9940
315/375 [========================>.....] - ETA: 0s - loss: 0.0209 - accuracy: 0.9939
321/375 [========================>.....] - ETA: 0s - loss: 0.0209 - accuracy: 0.9939
326/375 [=========================>....] - ETA: 0s - loss: 0.0210 - accuracy: 0.9939
333/375 [=========================>....] - ETA: 0s - loss: 0.0210 - accuracy: 0.9939
339/375 [==========================>...] - ETA: 0s - loss: 0.0214 - accuracy: 0.9937
345/375 [==========================>...] - ETA: 0s - loss: 0.0216 - accuracy: 0.9937
352/375 [===========================>..] - ETA: 0s - loss: 0.0217 - accuracy: 0.9936
359/375 [===========================>..] - ETA: 0s - loss: 0.0220 - accuracy: 0.9935
365/375 [============================>.] - ETA: 0s - loss: 0.0219 - accuracy: 0.9936
371/375 [============================>.] - ETA: 0s - loss: 0.0219 - accuracy: 0.9936
375/375 [==============================] - 3s 8ms/step - loss: 0.0219 - accuracy: 0.9936

375/375 [==============================] - 4s 11ms/step - loss: 0.0219 - accuracy: 0.9936 - val_loss: 0.0814 - val_accuracy: 0.9785 - lr: 0.0100
Epoch 8/20

  1/375 [..............................] - ETA: 3s - loss: 0.0016 - accuracy: 1.0000
  8/375 [..............................] - ETA: 3s - loss: 0.0096 - accuracy: 0.9971
 14/375 [>.............................] - ETA: 3s - loss: 0.0129 - accuracy: 0.9967
 20/375 [>.............................] - ETA: 3s - loss: 0.0123 - accuracy: 0.9965
 26/375 [=>............................] - ETA: 3s - loss: 0.0131 - accuracy: 0.9967
 32/375 [=>............................] - ETA: 3s - loss: 0.0129 - accuracy: 0.9971
 38/375 [==>...........................] - ETA: 2s - loss: 0.0138 - accuracy: 0.9967
 44/375 [==>...........................] - ETA: 2s - loss: 0.0138 - accuracy: 0.9968
 50/375 [===>..........................] - ETA: 2s - loss: 0.0143 - accuracy: 0.9967
 56/375 [===>..........................] - ETA: 2s - loss: 0.0147 - accuracy: 0.9965
 62/375 [===>..........................] - ETA: 2s - loss: 0.0154 - accuracy: 0.9961
 67/375 [====>.........................] - ETA: 2s - loss: 0.0153 - accuracy: 0.9962
 71/375 [====>.........................] - ETA: 2s - loss: 0.0158 - accuracy: 0.9960
 77/375 [=====>........................] - ETA: 2s - loss: 0.0161 - accuracy: 0.9957
 83/375 [=====>........................] - ETA: 2s - loss: 0.0162 - accuracy: 0.9958
 89/375 [======>.......................] - ETA: 2s - loss: 0.0168 - accuracy: 0.9955
 95/375 [======>.......................] - ETA: 2s - loss: 0.0177 - accuracy: 0.9954
101/375 [=======>......................] - ETA: 2s - loss: 0.0177 - accuracy: 0.9954
107/375 [=======>......................] - ETA: 2s - loss: 0.0179 - accuracy: 0.9954
113/375 [========>.....................] - ETA: 2s - loss: 0.0175 - accuracy: 0.9955
118/375 [========>.....................] - ETA: 2s - loss: 0.0170 - accuracy: 0.9957
123/375 [========>.....................] - ETA: 2s - loss: 0.0171 - accuracy: 0.9956
128/375 [=========>....................] - ETA: 2s - loss: 0.0170 - accuracy: 0.9955
133/375 [=========>....................] - ETA: 2s - loss: 0.0171 - accuracy: 0.9955
139/375 [==========>...................] - ETA: 2s - loss: 0.0172 - accuracy: 0.9953
144/375 [==========>...................] - ETA: 2s - loss: 0.0170 - accuracy: 0.9954
149/375 [==========>...................] - ETA: 2s - loss: 0.0169 - accuracy: 0.9954
154/375 [===========>..................] - ETA: 2s - loss: 0.0171 - accuracy: 0.9953
160/375 [===========>..................] - ETA: 2s - loss: 0.0176 - accuracy: 0.9953
166/375 [============>.................] - ETA: 2s - loss: 0.0180 - accuracy: 0.9952
171/375 [============>.................] - ETA: 2s - loss: 0.0182 - accuracy: 0.9951
177/375 [=============>................] - ETA: 1s - loss: 0.0185 - accuracy: 0.9951
183/375 [=============>................] - ETA: 1s - loss: 0.0183 - accuracy: 0.9951
188/375 [==============>...............] - ETA: 1s - loss: 0.0184 - accuracy: 0.9951
193/375 [==============>...............] - ETA: 1s - loss: 0.0184 - accuracy: 0.9952
199/375 [==============>...............] - ETA: 1s - loss: 0.0184 - accuracy: 0.9952
204/375 [===============>..............] - ETA: 1s - loss: 0.0182 - accuracy: 0.9953
210/375 [===============>..............] - ETA: 1s - loss: 0.0183 - accuracy: 0.9952
216/375 [================>.............] - ETA: 1s - loss: 0.0184 - accuracy: 0.9952
222/375 [================>.............] - ETA: 1s - loss: 0.0184 - accuracy: 0.9952
228/375 [=================>............] - ETA: 1s - loss: 0.0186 - accuracy: 0.9951
233/375 [=================>............] - ETA: 1s - loss: 0.0185 - accuracy: 0.9951
238/375 [==================>...........] - ETA: 1s - loss: 0.0185 - accuracy: 0.9950
243/375 [==================>...........] - ETA: 1s - loss: 0.0184 - accuracy: 0.9950
248/375 [==================>...........] - ETA: 1s - loss: 0.0185 - accuracy: 0.9950
254/375 [===================>..........] - ETA: 1s - loss: 0.0184 - accuracy: 0.9950
259/375 [===================>..........] - ETA: 1s - loss: 0.0184 - accuracy: 0.9951
265/375 [====================>.........] - ETA: 1s - loss: 0.0183 - accuracy: 0.9950
270/375 [====================>.........] - ETA: 1s - loss: 0.0181 - accuracy: 0.9951
275/375 [=====================>........] - ETA: 0s - loss: 0.0179 - accuracy: 0.9952
281/375 [=====================>........] - ETA: 0s - loss: 0.0178 - accuracy: 0.9952
287/375 [=====================>........] - ETA: 0s - loss: 0.0177 - accuracy: 0.9952
293/375 [======================>.......] - ETA: 0s - loss: 0.0179 - accuracy: 0.9952
299/375 [======================>.......] - ETA: 0s - loss: 0.0178 - accuracy: 0.9952
305/375 [=======================>......] - ETA: 0s - loss: 0.0178 - accuracy: 0.9952
311/375 [=======================>......] - ETA: 0s - loss: 0.0179 - accuracy: 0.9952
317/375 [========================>.....] - ETA: 0s - loss: 0.0178 - accuracy: 0.9952
323/375 [========================>.....] - ETA: 0s - loss: 0.0179 - accuracy: 0.9952
329/375 [=========================>....] - ETA: 0s - loss: 0.0179 - accuracy: 0.9951
334/375 [=========================>....] - ETA: 0s - loss: 0.0180 - accuracy: 0.9951
339/375 [==========================>...] - ETA: 0s - loss: 0.0178 - accuracy: 0.9952
345/375 [==========================>...] - ETA: 0s - loss: 0.0180 - accuracy: 0.9951
350/375 [===========================>..] - ETA: 0s - loss: 0.0179 - accuracy: 0.9951
355/375 [===========================>..] - ETA: 0s - loss: 0.0179 - accuracy: 0.9951
361/375 [===========================>..] - ETA: 0s - loss: 0.0179 - accuracy: 0.9951
365/375 [============================>.] - ETA: 0s - loss: 0.0180 - accuracy: 0.9950
370/375 [============================>.] - ETA: 0s - loss: 0.0182 - accuracy: 0.9950
375/375 [==============================] - 4s 10ms/step - loss: 0.0182 - accuracy: 0.9950

375/375 [==============================] - 5s 13ms/step - loss: 0.0182 - accuracy: 0.9950 - val_loss: 0.0816 - val_accuracy: 0.9792 - lr: 0.0100
Epoch 9/20

  1/375 [..............................] - ETA: 2s - loss: 0.0150 - accuracy: 0.9922
  7/375 [..............................] - ETA: 3s - loss: 0.0123 - accuracy: 0.9955
 13/375 [>.............................] - ETA: 3s - loss: 0.0152 - accuracy: 0.9946
 19/375 [>.............................] - ETA: 3s - loss: 0.0150 - accuracy: 0.9938
 26/375 [=>............................] - ETA: 2s - loss: 0.0139 - accuracy: 0.9946
 32/375 [=>............................] - ETA: 2s - loss: 0.0143 - accuracy: 0.9946
 39/375 [==>...........................] - ETA: 2s - loss: 0.0140 - accuracy: 0.9950
 46/375 [==>...........................] - ETA: 2s - loss: 0.0137 - accuracy: 0.9951
 52/375 [===>..........................] - ETA: 2s - loss: 0.0133 - accuracy: 0.9952
 59/375 [===>..........................] - ETA: 2s - loss: 0.0139 - accuracy: 0.9952
 65/375 [====>.........................] - ETA: 2s - loss: 0.0138 - accuracy: 0.9956
 71/375 [====>.........................] - ETA: 2s - loss: 0.0136 - accuracy: 0.9956
 77/375 [=====>........................] - ETA: 2s - loss: 0.0136 - accuracy: 0.9957
 81/375 [=====>........................] - ETA: 2s - loss: 0.0137 - accuracy: 0.9957
 87/375 [=====>........................] - ETA: 2s - loss: 0.0140 - accuracy: 0.9958
 94/375 [======>.......................] - ETA: 2s - loss: 0.0137 - accuracy: 0.9959
100/375 [=======>......................] - ETA: 2s - loss: 0.0135 - accuracy: 0.9960
106/375 [=======>......................] - ETA: 2s - loss: 0.0136 - accuracy: 0.9959
112/375 [=======>......................] - ETA: 2s - loss: 0.0139 - accuracy: 0.9958
118/375 [========>.....................] - ETA: 2s - loss: 0.0139 - accuracy: 0.9959
123/375 [========>.....................] - ETA: 2s - loss: 0.0142 - accuracy: 0.9957
129/375 [=========>....................] - ETA: 2s - loss: 0.0142 - accuracy: 0.9958
135/375 [=========>....................] - ETA: 2s - loss: 0.0139 - accuracy: 0.9960
141/375 [==========>...................] - ETA: 2s - loss: 0.0137 - accuracy: 0.9961
147/375 [==========>...................] - ETA: 2s - loss: 0.0136 - accuracy: 0.9961
153/375 [===========>..................] - ETA: 1s - loss: 0.0135 - accuracy: 0.9962
159/375 [===========>..................] - ETA: 1s - loss: 0.0142 - accuracy: 0.9960
165/375 [============>.................] - ETA: 1s - loss: 0.0140 - accuracy: 0.9961
171/375 [============>.................] - ETA: 1s - loss: 0.0140 - accuracy: 0.9961
177/375 [=============>................] - ETA: 1s - loss: 0.0142 - accuracy: 0.9960
183/375 [=============>................] - ETA: 1s - loss: 0.0143 - accuracy: 0.9959
189/375 [==============>...............] - ETA: 1s - loss: 0.0146 - accuracy: 0.9959
195/375 [==============>...............] - ETA: 1s - loss: 0.0144 - accuracy: 0.9960
201/375 [===============>..............] - ETA: 1s - loss: 0.0148 - accuracy: 0.9959
208/375 [===============>..............] - ETA: 1s - loss: 0.0150 - accuracy: 0.9959
214/375 [================>.............] - ETA: 1s - loss: 0.0152 - accuracy: 0.9959
220/375 [================>.............] - ETA: 1s - loss: 0.0149 - accuracy: 0.9960
226/375 [=================>............] - ETA: 1s - loss: 0.0151 - accuracy: 0.9960
233/375 [=================>............] - ETA: 1s - loss: 0.0150 - accuracy: 0.9960
239/375 [==================>...........] - ETA: 1s - loss: 0.0147 - accuracy: 0.9961
244/375 [==================>...........] - ETA: 1s - loss: 0.0149 - accuracy: 0.9960
249/375 [==================>...........] - ETA: 1s - loss: 0.0150 - accuracy: 0.9960
255/375 [===================>..........] - ETA: 1s - loss: 0.0152 - accuracy: 0.9959
260/375 [===================>..........] - ETA: 1s - loss: 0.0151 - accuracy: 0.9960
266/375 [====================>.........] - ETA: 0s - loss: 0.0149 - accuracy: 0.9960
272/375 [====================>.........] - ETA: 0s - loss: 0.0149 - accuracy: 0.9960
278/375 [=====================>........] - ETA: 0s - loss: 0.0151 - accuracy: 0.9960
284/375 [=====================>........] - ETA: 0s - loss: 0.0151 - accuracy: 0.9960
290/375 [======================>.......] - ETA: 0s - loss: 0.0151 - accuracy: 0.9959
296/375 [======================>.......] - ETA: 0s - loss: 0.0151 - accuracy: 0.9959
301/375 [=======================>......] - ETA: 0s - loss: 0.0150 - accuracy: 0.9960
307/375 [=======================>......] - ETA: 0s - loss: 0.0150 - accuracy: 0.9959
313/375 [========================>.....] - ETA: 0s - loss: 0.0149 - accuracy: 0.9960
319/375 [========================>.....] - ETA: 0s - loss: 0.0149 - accuracy: 0.9960
325/375 [=========================>....] - ETA: 0s - loss: 0.0151 - accuracy: 0.9960
331/375 [=========================>....] - ETA: 0s - loss: 0.0151 - accuracy: 0.9960
337/375 [=========================>....] - ETA: 0s - loss: 0.0151 - accuracy: 0.9960
344/375 [==========================>...] - ETA: 0s - loss: 0.0150 - accuracy: 0.9960
350/375 [===========================>..] - ETA: 0s - loss: 0.0150 - accuracy: 0.9960
356/375 [===========================>..] - ETA: 0s - loss: 0.0149 - accuracy: 0.9961
362/375 [===========================>..] - ETA: 0s - loss: 0.0150 - accuracy: 0.9960
368/375 [============================>.] - ETA: 0s - loss: 0.0149 - accuracy: 0.9960
374/375 [============================>.] - ETA: 0s - loss: 0.0151 - accuracy: 0.9960
375/375 [==============================] - 4s 10ms/step - loss: 0.0152 - accuracy: 0.9960

375/375 [==============================] - 5s 12ms/step - loss: 0.0152 - accuracy: 0.9960 - val_loss: 0.0811 - val_accuracy: 0.9791 - lr: 1.0000e-03
Epoch 10/20

  1/375 [..............................] - ETA: 3s - loss: 0.0148 - accuracy: 1.0000
  8/375 [..............................] - ETA: 2s - loss: 0.0172 - accuracy: 0.9971
 14/375 [>.............................] - ETA: 2s - loss: 0.0140 - accuracy: 0.9978
 21/375 [>.............................] - ETA: 2s - loss: 0.0139 - accuracy: 0.9967
 27/375 [=>............................] - ETA: 2s - loss: 0.0145 - accuracy: 0.9965
 33/375 [=>............................] - ETA: 2s - loss: 0.0156 - accuracy: 0.9962
 39/375 [==>...........................] - ETA: 2s - loss: 0.0161 - accuracy: 0.9958
 45/375 [==>...........................] - ETA: 2s - loss: 0.0155 - accuracy: 0.9960
 51/375 [===>..........................] - ETA: 2s - loss: 0.0173 - accuracy: 0.9957
 57/375 [===>..........................] - ETA: 2s - loss: 0.0169 - accuracy: 0.9960
 63/375 [====>.........................] - ETA: 2s - loss: 0.0160 - accuracy: 0.9963
 69/375 [====>.........................] - ETA: 2s - loss: 0.0156 - accuracy: 0.9964
 75/375 [=====>........................] - ETA: 2s - loss: 0.0157 - accuracy: 0.9964
 82/375 [=====>........................] - ETA: 2s - loss: 0.0156 - accuracy: 0.9961
 88/375 [======>.......................] - ETA: 2s - loss: 0.0157 - accuracy: 0.9960
 95/375 [======>.......................] - ETA: 2s - loss: 0.0157 - accuracy: 0.9961
101/375 [=======>......................] - ETA: 2s - loss: 0.0155 - accuracy: 0.9961
107/375 [=======>......................] - ETA: 2s - loss: 0.0157 - accuracy: 0.9961
113/375 [========>.....................] - ETA: 2s - loss: 0.0158 - accuracy: 0.9961
120/375 [========>.....................] - ETA: 2s - loss: 0.0156 - accuracy: 0.9960
123/375 [========>.....................] - ETA: 2s - loss: 0.0155 - accuracy: 0.9960
127/375 [=========>....................] - ETA: 2s - loss: 0.0159 - accuracy: 0.9960
134/375 [=========>....................] - ETA: 2s - loss: 0.0159 - accuracy: 0.9960
140/375 [==========>...................] - ETA: 2s - loss: 0.0156 - accuracy: 0.9961
146/375 [==========>...................] - ETA: 2s - loss: 0.0157 - accuracy: 0.9959
152/375 [===========>..................] - ETA: 2s - loss: 0.0160 - accuracy: 0.9958
158/375 [===========>..................] - ETA: 1s - loss: 0.0160 - accuracy: 0.9957
165/375 [============>.................] - ETA: 1s - loss: 0.0156 - accuracy: 0.9959
171/375 [============>.................] - ETA: 1s - loss: 0.0158 - accuracy: 0.9958
177/375 [=============>................] - ETA: 1s - loss: 0.0157 - accuracy: 0.9958
183/375 [=============>................] - ETA: 1s - loss: 0.0158 - accuracy: 0.9957
188/375 [==============>...............] - ETA: 1s - loss: 0.0158 - accuracy: 0.9957
194/375 [==============>...............] - ETA: 1s - loss: 0.0157 - accuracy: 0.9957
200/375 [===============>..............] - ETA: 1s - loss: 0.0155 - accuracy: 0.9958
206/375 [===============>..............] - ETA: 1s - loss: 0.0157 - accuracy: 0.9958
212/375 [===============>..............] - ETA: 1s - loss: 0.0160 - accuracy: 0.9957
218/375 [================>.............] - ETA: 1s - loss: 0.0158 - accuracy: 0.9957
224/375 [================>.............] - ETA: 1s - loss: 0.0157 - accuracy: 0.9957
230/375 [=================>............] - ETA: 1s - loss: 0.0155 - accuracy: 0.9959
236/375 [=================>............] - ETA: 1s - loss: 0.0153 - accuracy: 0.9959
242/375 [==================>...........] - ETA: 1s - loss: 0.0154 - accuracy: 0.9959
248/375 [==================>...........] - ETA: 1s - loss: 0.0154 - accuracy: 0.9959
254/375 [===================>..........] - ETA: 1s - loss: 0.0152 - accuracy: 0.9959
260/375 [===================>..........] - ETA: 1s - loss: 0.0152 - accuracy: 0.9959
266/375 [====================>.........] - ETA: 0s - loss: 0.0153 - accuracy: 0.9960
272/375 [====================>.........] - ETA: 0s - loss: 0.0153 - accuracy: 0.9960
278/375 [=====================>........] - ETA: 0s - loss: 0.0152 - accuracy: 0.9960
284/375 [=====================>........] - ETA: 0s - loss: 0.0152 - accuracy: 0.9960
291/375 [======================>.......] - ETA: 0s - loss: 0.0153 - accuracy: 0.9960
297/375 [======================>.......] - ETA: 0s - loss: 0.0152 - accuracy: 0.9960
303/375 [=======================>......] - ETA: 0s - loss: 0.0152 - accuracy: 0.9960
309/375 [=======================>......] - ETA: 0s - loss: 0.0153 - accuracy: 0.9960
315/375 [========================>.....] - ETA: 0s - loss: 0.0153 - accuracy: 0.9960
321/375 [========================>.....] - ETA: 0s - loss: 0.0152 - accuracy: 0.9961
327/375 [=========================>....] - ETA: 0s - loss: 0.0151 - accuracy: 0.9961
333/375 [=========================>....] - ETA: 0s - loss: 0.0150 - accuracy: 0.9961
338/375 [==========================>...] - ETA: 0s - loss: 0.0149 - accuracy: 0.9962
341/375 [==========================>...] - ETA: 0s - loss: 0.0150 - accuracy: 0.9961
347/375 [==========================>...] - ETA: 0s - loss: 0.0149 - accuracy: 0.9961
353/375 [===========================>..] - ETA: 0s - loss: 0.0148 - accuracy: 0.9962
359/375 [===========================>..] - ETA: 0s - loss: 0.0149 - accuracy: 0.9961
365/375 [============================>.] - ETA: 0s - loss: 0.0148 - accuracy: 0.9962
371/375 [============================>.] - ETA: 0s - loss: 0.0148 - accuracy: 0.9962
375/375 [==============================] - 3s 9ms/step - loss: 0.0148 - accuracy: 0.9962

375/375 [==============================] - 4s 11ms/step - loss: 0.0148 - accuracy: 0.9962 - val_loss: 0.0811 - val_accuracy: 0.9793 - lr: 1.0000e-03
Epoch 11/20

  1/375 [..............................] - ETA: 12s - loss: 0.0120 - accuracy: 1.0000
  8/375 [..............................] - ETA: 2s - loss: 0.0118 - accuracy: 0.9980 
 14/375 [>.............................] - ETA: 2s - loss: 0.0147 - accuracy: 0.9961
 18/375 [>.............................] - ETA: 3s - loss: 0.0170 - accuracy: 0.9957
 25/375 [=>............................] - ETA: 3s - loss: 0.0155 - accuracy: 0.9959
 32/375 [=>............................] - ETA: 3s - loss: 0.0159 - accuracy: 0.9956
 38/375 [==>...........................] - ETA: 2s - loss: 0.0184 - accuracy: 0.9951
 45/375 [==>...........................] - ETA: 2s - loss: 0.0166 - accuracy: 0.9957
 52/375 [===>..........................] - ETA: 2s - loss: 0.0159 - accuracy: 0.9958
 57/375 [===>..........................] - ETA: 2s - loss: 0.0154 - accuracy: 0.9959
 63/375 [====>.........................] - ETA: 2s - loss: 0.0150 - accuracy: 0.9962
 69/375 [====>.........................] - ETA: 2s - loss: 0.0151 - accuracy: 0.9959
 74/375 [====>.........................] - ETA: 2s - loss: 0.0147 - accuracy: 0.9961
 80/375 [=====>........................] - ETA: 2s - loss: 0.0158 - accuracy: 0.9956
 86/375 [=====>........................] - ETA: 2s - loss: 0.0157 - accuracy: 0.9955
 92/375 [======>.......................] - ETA: 2s - loss: 0.0151 - accuracy: 0.9958
 98/375 [======>.......................] - ETA: 2s - loss: 0.0148 - accuracy: 0.9959
104/375 [=======>......................] - ETA: 2s - loss: 0.0145 - accuracy: 0.9961
110/375 [=======>......................] - ETA: 2s - loss: 0.0153 - accuracy: 0.9959
116/375 [========>.....................] - ETA: 2s - loss: 0.0153 - accuracy: 0.9958
122/375 [========>.....................] - ETA: 2s - loss: 0.0151 - accuracy: 0.9959
129/375 [=========>....................] - ETA: 2s - loss: 0.0154 - accuracy: 0.9957
135/375 [=========>....................] - ETA: 2s - loss: 0.0154 - accuracy: 0.9957
141/375 [==========>...................] - ETA: 2s - loss: 0.0152 - accuracy: 0.9958
148/375 [==========>...................] - ETA: 2s - loss: 0.0150 - accuracy: 0.9959
153/375 [===========>..................] - ETA: 1s - loss: 0.0150 - accuracy: 0.9959
159/375 [===========>..................] - ETA: 1s - loss: 0.0153 - accuracy: 0.9957
164/375 [============>.................] - ETA: 1s - loss: 0.0152 - accuracy: 0.9957
169/375 [============>.................] - ETA: 1s - loss: 0.0151 - accuracy: 0.9958
174/375 [============>.................] - ETA: 1s - loss: 0.0149 - accuracy: 0.9959
180/375 [=============>................] - ETA: 1s - loss: 0.0147 - accuracy: 0.9960
186/375 [=============>................] - ETA: 1s - loss: 0.0145 - accuracy: 0.9961
191/375 [==============>...............] - ETA: 1s - loss: 0.0145 - accuracy: 0.9960
197/375 [==============>...............] - ETA: 1s - loss: 0.0144 - accuracy: 0.9961
203/375 [===============>..............] - ETA: 1s - loss: 0.0143 - accuracy: 0.9962
209/375 [===============>..............] - ETA: 1s - loss: 0.0142 - accuracy: 0.9963
216/375 [================>.............] - ETA: 1s - loss: 0.0140 - accuracy: 0.9964
222/375 [================>.............] - ETA: 1s - loss: 0.0139 - accuracy: 0.9964
228/375 [=================>............] - ETA: 1s - loss: 0.0139 - accuracy: 0.9964
235/375 [=================>............] - ETA: 1s - loss: 0.0137 - accuracy: 0.9964
241/375 [==================>...........] - ETA: 1s - loss: 0.0136 - accuracy: 0.9965
247/375 [==================>...........] - ETA: 1s - loss: 0.0136 - accuracy: 0.9964
254/375 [===================>..........] - ETA: 1s - loss: 0.0135 - accuracy: 0.9965
259/375 [===================>..........] - ETA: 1s - loss: 0.0137 - accuracy: 0.9964
266/375 [====================>.........] - ETA: 0s - loss: 0.0140 - accuracy: 0.9964
272/375 [====================>.........] - ETA: 0s - loss: 0.0141 - accuracy: 0.9964
278/375 [=====================>........] - ETA: 0s - loss: 0.0140 - accuracy: 0.9964
285/375 [=====================>........] - ETA: 0s - loss: 0.0138 - accuracy: 0.9965
292/375 [======================>.......] - ETA: 0s - loss: 0.0141 - accuracy: 0.9964
298/375 [======================>.......] - ETA: 0s - loss: 0.0142 - accuracy: 0.9964
304/375 [=======================>......] - ETA: 0s - loss: 0.0142 - accuracy: 0.9964
310/375 [=======================>......] - ETA: 0s - loss: 0.0143 - accuracy: 0.9963
316/375 [========================>.....] - ETA: 0s - loss: 0.0141 - accuracy: 0.9964
322/375 [========================>.....] - ETA: 0s - loss: 0.0142 - accuracy: 0.9963
327/375 [=========================>....] - ETA: 0s - loss: 0.0142 - accuracy: 0.9963
333/375 [=========================>....] - ETA: 0s - loss: 0.0141 - accuracy: 0.9963
339/375 [==========================>...] - ETA: 0s - loss: 0.0140 - accuracy: 0.9964
346/375 [==========================>...] - ETA: 0s - loss: 0.0141 - accuracy: 0.9964
353/375 [===========================>..] - ETA: 0s - loss: 0.0142 - accuracy: 0.9964
360/375 [===========================>..] - ETA: 0s - loss: 0.0144 - accuracy: 0.9963
366/375 [============================>.] - ETA: 0s - loss: 0.0145 - accuracy: 0.9962
373/375 [============================>.] - ETA: 0s - loss: 0.0145 - accuracy: 0.9963
375/375 [==============================] - 3s 9ms/step - loss: 0.0144 - accuracy: 0.9962

375/375 [==============================] - 4s 11ms/step - loss: 0.0144 - accuracy: 0.9962 - val_loss: 0.0812 - val_accuracy: 0.9794 - lr: 1.0000e-04
Epoch 12/20

  1/375 [..............................] - ETA: 3s - loss: 0.0025 - accuracy: 1.0000
  8/375 [..............................] - ETA: 3s - loss: 0.0157 - accuracy: 0.9951
 14/375 [>.............................] - ETA: 3s - loss: 0.0147 - accuracy: 0.9955
 20/375 [>.............................] - ETA: 3s - loss: 0.0162 - accuracy: 0.9949
 26/375 [=>............................] - ETA: 3s - loss: 0.0170 - accuracy: 0.9952
 33/375 [=>............................] - ETA: 2s - loss: 0.0155 - accuracy: 0.9957
 39/375 [==>...........................] - ETA: 2s - loss: 0.0150 - accuracy: 0.9958
 45/375 [==>...........................] - ETA: 2s - loss: 0.0145 - accuracy: 0.9960
 51/375 [===>..........................] - ETA: 2s - loss: 0.0169 - accuracy: 0.9956
 57/375 [===>..........................] - ETA: 2s - loss: 0.0162 - accuracy: 0.9958
 63/375 [====>.........................] - ETA: 2s - loss: 0.0165 - accuracy: 0.9958
 69/375 [====>.........................] - ETA: 2s - loss: 0.0168 - accuracy: 0.9958
 75/375 [=====>........................] - ETA: 2s - loss: 0.0163 - accuracy: 0.9958
 81/375 [=====>........................] - ETA: 2s - loss: 0.0157 - accuracy: 0.9960
 87/375 [=====>........................] - ETA: 2s - loss: 0.0159 - accuracy: 0.9958
 93/375 [======>.......................] - ETA: 2s - loss: 0.0157 - accuracy: 0.9959
 99/375 [======>.......................] - ETA: 2s - loss: 0.0155 - accuracy: 0.9959
106/375 [=======>......................] - ETA: 2s - loss: 0.0158 - accuracy: 0.9959
112/375 [=======>......................] - ETA: 2s - loss: 0.0155 - accuracy: 0.9960
118/375 [========>.....................] - ETA: 2s - loss: 0.0158 - accuracy: 0.9959
124/375 [========>.....................] - ETA: 2s - loss: 0.0159 - accuracy: 0.9959
131/375 [=========>....................] - ETA: 2s - loss: 0.0160 - accuracy: 0.9958
137/375 [=========>....................] - ETA: 2s - loss: 0.0161 - accuracy: 0.9957
143/375 [==========>...................] - ETA: 2s - loss: 0.0163 - accuracy: 0.9956
149/375 [==========>...................] - ETA: 1s - loss: 0.0162 - accuracy: 0.9956
155/375 [===========>..................] - ETA: 1s - loss: 0.0163 - accuracy: 0.9957
162/375 [===========>..................] - ETA: 1s - loss: 0.0161 - accuracy: 0.9957
168/375 [============>.................] - ETA: 1s - loss: 0.0158 - accuracy: 0.9958
175/375 [=============>................] - ETA: 1s - loss: 0.0157 - accuracy: 0.9959
182/375 [=============>................] - ETA: 1s - loss: 0.0158 - accuracy: 0.9958
188/375 [==============>...............] - ETA: 1s - loss: 0.0154 - accuracy: 0.9959
194/375 [==============>...............] - ETA: 1s - loss: 0.0153 - accuracy: 0.9959
200/375 [===============>..............] - ETA: 1s - loss: 0.0153 - accuracy: 0.9959
206/375 [===============>..............] - ETA: 1s - loss: 0.0152 - accuracy: 0.9960
212/375 [===============>..............] - ETA: 1s - loss: 0.0151 - accuracy: 0.9961
219/375 [================>.............] - ETA: 1s - loss: 0.0154 - accuracy: 0.9960
226/375 [=================>............] - ETA: 1s - loss: 0.0152 - accuracy: 0.9961
232/375 [=================>............] - ETA: 1s - loss: 0.0152 - accuracy: 0.9961
238/375 [==================>...........] - ETA: 1s - loss: 0.0150 - accuracy: 0.9962
244/375 [==================>...........] - ETA: 1s - loss: 0.0151 - accuracy: 0.9963
250/375 [===================>..........] - ETA: 1s - loss: 0.0150 - accuracy: 0.9963
256/375 [===================>..........] - ETA: 1s - loss: 0.0149 - accuracy: 0.9963
262/375 [===================>..........] - ETA: 0s - loss: 0.0154 - accuracy: 0.9960
268/375 [====================>.........] - ETA: 0s - loss: 0.0154 - accuracy: 0.9961
274/375 [====================>.........] - ETA: 0s - loss: 0.0152 - accuracy: 0.9961
280/375 [=====================>........] - ETA: 0s - loss: 0.0152 - accuracy: 0.9960
285/375 [=====================>........] - ETA: 0s - loss: 0.0150 - accuracy: 0.9961
291/375 [======================>.......] - ETA: 0s - loss: 0.0151 - accuracy: 0.9961
297/375 [======================>.......] - ETA: 0s - loss: 0.0149 - accuracy: 0.9962
302/375 [=======================>......] - ETA: 0s - loss: 0.0147 - accuracy: 0.9962
308/375 [=======================>......] - ETA: 0s - loss: 0.0146 - accuracy: 0.9962
315/375 [========================>.....] - ETA: 0s - loss: 0.0145 - accuracy: 0.9963
321/375 [========================>.....] - ETA: 0s - loss: 0.0146 - accuracy: 0.9963
327/375 [=========================>....] - ETA: 0s - loss: 0.0145 - accuracy: 0.9963
333/375 [=========================>....] - ETA: 0s - loss: 0.0146 - accuracy: 0.9963
339/375 [==========================>...] - ETA: 0s - loss: 0.0145 - accuracy: 0.9963
345/375 [==========================>...] - ETA: 0s - loss: 0.0145 - accuracy: 0.9963
351/375 [===========================>..] - ETA: 0s - loss: 0.0146 - accuracy: 0.9963
357/375 [===========================>..] - ETA: 0s - loss: 0.0145 - accuracy: 0.9963
363/375 [============================>.] - ETA: 0s - loss: 0.0145 - accuracy: 0.9963
369/375 [============================>.] - ETA: 0s - loss: 0.0145 - accuracy: 0.9963
375/375 [==============================] - 3s 9ms/step - loss: 0.0144 - accuracy: 0.9963

375/375 [==============================] - 4s 11ms/step - loss: 0.0144 - accuracy: 0.9963 - val_loss: 0.0812 - val_accuracy: 0.9794 - lr: 1.0000e-05
Epoch 13/20

  1/375 [..............................] - ETA: 3s - loss: 0.0202 - accuracy: 0.9922
  8/375 [..............................] - ETA: 3s - loss: 0.0109 - accuracy: 0.9971
 14/375 [>.............................] - ETA: 3s - loss: 0.0106 - accuracy: 0.9961
 20/375 [>.............................] - ETA: 3s - loss: 0.0109 - accuracy: 0.9969
 26/375 [=>............................] - ETA: 3s - loss: 0.0111 - accuracy: 0.9970
 31/375 [=>............................] - ETA: 3s - loss: 0.0118 - accuracy: 0.9967
 36/375 [=>............................] - ETA: 3s - loss: 0.0114 - accuracy: 0.9970
 42/375 [==>...........................] - ETA: 3s - loss: 0.0117 - accuracy: 0.9970
 48/375 [==>...........................] - ETA: 3s - loss: 0.0110 - accuracy: 0.9972
 55/375 [===>..........................] - ETA: 2s - loss: 0.0126 - accuracy: 0.9967
 62/375 [===>..........................] - ETA: 2s - loss: 0.0126 - accuracy: 0.9967
 69/375 [====>.........................] - ETA: 2s - loss: 0.0121 - accuracy: 0.9971
 76/375 [=====>........................] - ETA: 2s - loss: 0.0125 - accuracy: 0.9968
 83/375 [=====>........................] - ETA: 2s - loss: 0.0126 - accuracy: 0.9966
 90/375 [======>.......................] - ETA: 2s - loss: 0.0127 - accuracy: 0.9965
 97/375 [======>.......................] - ETA: 2s - loss: 0.0136 - accuracy: 0.9963
103/375 [=======>......................] - ETA: 2s - loss: 0.0133 - accuracy: 0.9964
109/375 [=======>......................] - ETA: 2s - loss: 0.0131 - accuracy: 0.9966
114/375 [========>.....................] - ETA: 2s - loss: 0.0133 - accuracy: 0.9966
120/375 [========>.....................] - ETA: 2s - loss: 0.0136 - accuracy: 0.9965
126/375 [=========>....................] - ETA: 2s - loss: 0.0138 - accuracy: 0.9965
133/375 [=========>....................] - ETA: 2s - loss: 0.0139 - accuracy: 0.9965
140/375 [==========>...................] - ETA: 2s - loss: 0.0137 - accuracy: 0.9967
147/375 [==========>...................] - ETA: 2s - loss: 0.0136 - accuracy: 0.9967
154/375 [===========>..................] - ETA: 1s - loss: 0.0136 - accuracy: 0.9967
161/375 [===========>..................] - ETA: 1s - loss: 0.0133 - accuracy: 0.9968
168/375 [============>.................] - ETA: 1s - loss: 0.0132 - accuracy: 0.9968
174/375 [============>.................] - ETA: 1s - loss: 0.0130 - accuracy: 0.9969
181/375 [=============>................] - ETA: 1s - loss: 0.0133 - accuracy: 0.9967
188/375 [==============>...............] - ETA: 1s - loss: 0.0132 - accuracy: 0.9968
195/375 [==============>...............] - ETA: 1s - loss: 0.0135 - accuracy: 0.9967
202/375 [===============>..............] - ETA: 1s - loss: 0.0137 - accuracy: 0.9966
208/375 [===============>..............] - ETA: 1s - loss: 0.0138 - accuracy: 0.9965
214/375 [================>.............] - ETA: 1s - loss: 0.0138 - accuracy: 0.9965
221/375 [================>.............] - ETA: 1s - loss: 0.0143 - accuracy: 0.9963
226/375 [=================>............] - ETA: 1s - loss: 0.0142 - accuracy: 0.9963
232/375 [=================>............] - ETA: 1s - loss: 0.0140 - accuracy: 0.9964
238/375 [==================>...........] - ETA: 1s - loss: 0.0142 - accuracy: 0.9963
245/375 [==================>...........] - ETA: 1s - loss: 0.0142 - accuracy: 0.9963
252/375 [===================>..........] - ETA: 1s - loss: 0.0141 - accuracy: 0.9963
259/375 [===================>..........] - ETA: 1s - loss: 0.0143 - accuracy: 0.9963
266/375 [====================>.........] - ETA: 0s - loss: 0.0143 - accuracy: 0.9963
272/375 [====================>.........] - ETA: 0s - loss: 0.0143 - accuracy: 0.9963
279/375 [=====================>........] - ETA: 0s - loss: 0.0146 - accuracy: 0.9963
286/375 [=====================>........] - ETA: 0s - loss: 0.0145 - accuracy: 0.9963
293/375 [======================>.......] - ETA: 0s - loss: 0.0146 - accuracy: 0.9963
300/375 [=======================>......] - ETA: 0s - loss: 0.0147 - accuracy: 0.9963
307/375 [=======================>......] - ETA: 0s - loss: 0.0147 - accuracy: 0.9963
314/375 [========================>.....] - ETA: 0s - loss: 0.0147 - accuracy: 0.9963
321/375 [========================>.....] - ETA: 0s - loss: 0.0148 - accuracy: 0.9963
328/375 [=========================>....] - ETA: 0s - loss: 0.0147 - accuracy: 0.9964
335/375 [=========================>....] - ETA: 0s - loss: 0.0146 - accuracy: 0.9964
342/375 [==========================>...] - ETA: 0s - loss: 0.0145 - accuracy: 0.9963
349/375 [==========================>...] - ETA: 0s - loss: 0.0144 - accuracy: 0.9964
356/375 [===========================>..] - ETA: 0s - loss: 0.0144 - accuracy: 0.9964
363/375 [============================>.] - ETA: 0s - loss: 0.0144 - accuracy: 0.9964
370/375 [============================>.] - ETA: 0s - loss: 0.0145 - accuracy: 0.9963
375/375 [==============================] - 3s 8ms/step - loss: 0.0144 - accuracy: 0.9963

375/375 [==============================] - 4s 11ms/step - loss: 0.0144 - accuracy: 0.9963 - val_loss: 0.0812 - val_accuracy: 0.9794 - lr: 1.0000e-06
Epoch 14/20

  1/375 [..............................] - ETA: 2s - loss: 0.0056 - accuracy: 1.0000
  7/375 [..............................] - ETA: 3s - loss: 0.0180 - accuracy: 0.9933
 13/375 [>.............................] - ETA: 3s - loss: 0.0141 - accuracy: 0.9952
 20/375 [>.............................] - ETA: 3s - loss: 0.0121 - accuracy: 0.9961
 27/375 [=>............................] - ETA: 2s - loss: 0.0129 - accuracy: 0.9959
 33/375 [=>............................] - ETA: 2s - loss: 0.0127 - accuracy: 0.9960
 40/375 [==>...........................] - ETA: 2s - loss: 0.0133 - accuracy: 0.9961
 46/375 [==>...........................] - ETA: 2s - loss: 0.0151 - accuracy: 0.9959
 52/375 [===>..........................] - ETA: 2s - loss: 0.0157 - accuracy: 0.9956
 59/375 [===>..........................] - ETA: 2s - loss: 0.0158 - accuracy: 0.9958
 65/375 [====>.........................] - ETA: 2s - loss: 0.0154 - accuracy: 0.9960
 71/375 [====>.........................] - ETA: 2s - loss: 0.0149 - accuracy: 0.9963
 78/375 [=====>........................] - ETA: 2s - loss: 0.0146 - accuracy: 0.9962
 85/375 [=====>........................] - ETA: 2s - loss: 0.0144 - accuracy: 0.9961
 92/375 [======>.......................] - ETA: 2s - loss: 0.0153 - accuracy: 0.9960
 99/375 [======>.......................] - ETA: 2s - loss: 0.0148 - accuracy: 0.9962
106/375 [=======>......................] - ETA: 2s - loss: 0.0152 - accuracy: 0.9962
113/375 [========>.....................] - ETA: 2s - loss: 0.0149 - accuracy: 0.9963
120/375 [========>.....................] - ETA: 2s - loss: 0.0145 - accuracy: 0.9964
127/375 [=========>....................] - ETA: 2s - loss: 0.0142 - accuracy: 0.9964
134/375 [=========>....................] - ETA: 1s - loss: 0.0145 - accuracy: 0.9963
141/375 [==========>...................] - ETA: 1s - loss: 0.0141 - accuracy: 0.9964
148/375 [==========>...................] - ETA: 1s - loss: 0.0138 - accuracy: 0.9965
155/375 [===========>..................] - ETA: 1s - loss: 0.0139 - accuracy: 0.9964
162/375 [===========>..................] - ETA: 1s - loss: 0.0138 - accuracy: 0.9965
169/375 [============>.................] - ETA: 1s - loss: 0.0140 - accuracy: 0.9964
176/375 [=============>................] - ETA: 1s - loss: 0.0139 - accuracy: 0.9964
183/375 [=============>................] - ETA: 1s - loss: 0.0139 - accuracy: 0.9964
190/375 [==============>...............] - ETA: 1s - loss: 0.0141 - accuracy: 0.9963
197/375 [==============>...............] - ETA: 1s - loss: 0.0140 - accuracy: 0.9964
203/375 [===============>..............] - ETA: 1s - loss: 0.0142 - accuracy: 0.9963
209/375 [===============>..............] - ETA: 1s - loss: 0.0144 - accuracy: 0.9962
216/375 [================>.............] - ETA: 1s - loss: 0.0146 - accuracy: 0.9962
223/375 [================>.............] - ETA: 1s - loss: 0.0146 - accuracy: 0.9962
230/375 [=================>............] - ETA: 1s - loss: 0.0145 - accuracy: 0.9963
237/375 [=================>............] - ETA: 1s - loss: 0.0145 - accuracy: 0.9963
244/375 [==================>...........] - ETA: 1s - loss: 0.0143 - accuracy: 0.9963
251/375 [===================>..........] - ETA: 1s - loss: 0.0145 - accuracy: 0.9962
255/375 [===================>..........] - ETA: 0s - loss: 0.0145 - accuracy: 0.9962
262/375 [===================>..........] - ETA: 0s - loss: 0.0149 - accuracy: 0.9961
269/375 [====================>.........] - ETA: 0s - loss: 0.0147 - accuracy: 0.9961
276/375 [=====================>........] - ETA: 0s - loss: 0.0149 - accuracy: 0.9961
283/375 [=====================>........] - ETA: 0s - loss: 0.0147 - accuracy: 0.9962
289/375 [======================>.......] - ETA: 0s - loss: 0.0148 - accuracy: 0.9961
296/375 [======================>.......] - ETA: 0s - loss: 0.0151 - accuracy: 0.9961
303/375 [=======================>......] - ETA: 0s - loss: 0.0152 - accuracy: 0.9961
310/375 [=======================>......] - ETA: 0s - loss: 0.0150 - accuracy: 0.9961
317/375 [========================>.....] - ETA: 0s - loss: 0.0149 - accuracy: 0.9962
324/375 [========================>.....] - ETA: 0s - loss: 0.0148 - accuracy: 0.9962
331/375 [=========================>....] - ETA: 0s - loss: 0.0147 - accuracy: 0.9962
338/375 [==========================>...] - ETA: 0s - loss: 0.0146 - accuracy: 0.9962
345/375 [==========================>...] - ETA: 0s - loss: 0.0145 - accuracy: 0.9963
352/375 [===========================>..] - ETA: 0s - loss: 0.0144 - accuracy: 0.9963
359/375 [===========================>..] - ETA: 0s - loss: 0.0144 - accuracy: 0.9963
366/375 [============================>.] - ETA: 0s - loss: 0.0145 - accuracy: 0.9963
373/375 [============================>.] - ETA: 0s - loss: 0.0144 - accuracy: 0.9963
375/375 [==============================] - 3s 8ms/step - loss: 0.0144 - accuracy: 0.9963

375/375 [==============================] - 4s 10ms/step - loss: 0.0144 - accuracy: 0.9963 - val_loss: 0.0812 - val_accuracy: 0.9794 - lr: 1.0000e-07
Epoch 15/20

  1/375 [..............................] - ETA: 2s - loss: 0.0038 - accuracy: 1.0000
  7/375 [..............................] - ETA: 3s - loss: 0.0116 - accuracy: 0.9978
 13/375 [>.............................] - ETA: 3s - loss: 0.0117 - accuracy: 0.9970
 19/375 [>.............................] - ETA: 3s - loss: 0.0136 - accuracy: 0.9959
 26/375 [=>............................] - ETA: 3s - loss: 0.0139 - accuracy: 0.9961
 33/375 [=>............................] - ETA: 3s - loss: 0.0138 - accuracy: 0.9964
 38/375 [==>...........................] - ETA: 3s - loss: 0.0146 - accuracy: 0.9963
 44/375 [==>...........................] - ETA: 3s - loss: 0.0150 - accuracy: 0.9963
 50/375 [===>..........................] - ETA: 3s - loss: 0.0146 - accuracy: 0.9964
 57/375 [===>..........................] - ETA: 2s - loss: 0.0151 - accuracy: 0.9960
 64/375 [====>.........................] - ETA: 2s - loss: 0.0155 - accuracy: 0.9955
 71/375 [====>.........................] - ETA: 2s - loss: 0.0156 - accuracy: 0.9954
 78/375 [=====>........................] - ETA: 2s - loss: 0.0159 - accuracy: 0.9953
 85/375 [=====>........................] - ETA: 2s - loss: 0.0159 - accuracy: 0.9953
 92/375 [======>.......................] - ETA: 2s - loss: 0.0161 - accuracy: 0.9953
 99/375 [======>.......................] - ETA: 2s - loss: 0.0161 - accuracy: 0.9954
106/375 [=======>......................] - ETA: 2s - loss: 0.0157 - accuracy: 0.9955
113/375 [========>.....................] - ETA: 2s - loss: 0.0161 - accuracy: 0.9955
120/375 [========>.....................] - ETA: 2s - loss: 0.0157 - accuracy: 0.9956
127/375 [=========>....................] - ETA: 2s - loss: 0.0154 - accuracy: 0.9958
134/375 [=========>....................] - ETA: 2s - loss: 0.0153 - accuracy: 0.9959
140/375 [==========>...................] - ETA: 1s - loss: 0.0150 - accuracy: 0.9960
147/375 [==========>...................] - ETA: 1s - loss: 0.0146 - accuracy: 0.9962
154/375 [===========>..................] - ETA: 1s - loss: 0.0150 - accuracy: 0.9961
161/375 [===========>..................] - ETA: 1s - loss: 0.0151 - accuracy: 0.9961
168/375 [============>.................] - ETA: 1s - loss: 0.0156 - accuracy: 0.9959
174/375 [============>.................] - ETA: 1s - loss: 0.0154 - accuracy: 0.9960
181/375 [=============>................] - ETA: 1s - loss: 0.0153 - accuracy: 0.9960
188/375 [==============>...............] - ETA: 1s - loss: 0.0152 - accuracy: 0.9960
195/375 [==============>...............] - ETA: 1s - loss: 0.0150 - accuracy: 0.9961
202/375 [===============>..............] - ETA: 1s - loss: 0.0151 - accuracy: 0.9959
209/375 [===============>..............] - ETA: 1s - loss: 0.0149 - accuracy: 0.9960
216/375 [================>.............] - ETA: 1s - loss: 0.0147 - accuracy: 0.9961
223/375 [================>.............] - ETA: 1s - loss: 0.0148 - accuracy: 0.9961
230/375 [=================>............] - ETA: 1s - loss: 0.0147 - accuracy: 0.9962
237/375 [=================>............] - ETA: 1s - loss: 0.0145 - accuracy: 0.9963
244/375 [==================>...........] - ETA: 1s - loss: 0.0143 - accuracy: 0.9964
251/375 [===================>..........] - ETA: 1s - loss: 0.0142 - accuracy: 0.9964
258/375 [===================>..........] - ETA: 0s - loss: 0.0142 - accuracy: 0.9965
265/375 [====================>.........] - ETA: 0s - loss: 0.0142 - accuracy: 0.9965
271/375 [====================>.........] - ETA: 0s - loss: 0.0142 - accuracy: 0.9965
278/375 [=====================>........] - ETA: 0s - loss: 0.0142 - accuracy: 0.9965
285/375 [=====================>........] - ETA: 0s - loss: 0.0142 - accuracy: 0.9965
287/375 [=====================>........] - ETA: 0s - loss: 0.0143 - accuracy: 0.9965
292/375 [======================>.......] - ETA: 0s - loss: 0.0143 - accuracy: 0.9965
299/375 [======================>.......] - ETA: 0s - loss: 0.0142 - accuracy: 0.9964
306/375 [=======================>......] - ETA: 0s - loss: 0.0142 - accuracy: 0.9964
313/375 [========================>.....] - ETA: 0s - loss: 0.0141 - accuracy: 0.9964
320/375 [========================>.....] - ETA: 0s - loss: 0.0142 - accuracy: 0.9964
327/375 [=========================>....] - ETA: 0s - loss: 0.0143 - accuracy: 0.9964
334/375 [=========================>....] - ETA: 0s - loss: 0.0143 - accuracy: 0.9964
341/375 [==========================>...] - ETA: 0s - loss: 0.0142 - accuracy: 0.9964
348/375 [==========================>...] - ETA: 0s - loss: 0.0142 - accuracy: 0.9964
355/375 [===========================>..] - ETA: 0s - loss: 0.0144 - accuracy: 0.9963
362/375 [===========================>..] - ETA: 0s - loss: 0.0143 - accuracy: 0.9963
369/375 [============================>.] - ETA: 0s - loss: 0.0145 - accuracy: 0.9963
375/375 [==============================] - 3s 8ms/step - loss: 0.0144 - accuracy: 0.9963

375/375 [==============================] - 4s 11ms/step - loss: 0.0144 - accuracy: 0.9963 - val_loss: 0.0812 - val_accuracy: 0.9794 - lr: 1.0000e-08
Epoch 16/20

  1/375 [..............................] - ETA: 3s - loss: 0.0069 - accuracy: 1.0000
  7/375 [..............................] - ETA: 3s - loss: 0.0064 - accuracy: 1.0000
 14/375 [>.............................] - ETA: 3s - loss: 0.0116 - accuracy: 0.9978
 20/375 [>.............................] - ETA: 3s - loss: 0.0111 - accuracy: 0.9980
 27/375 [=>............................] - ETA: 2s - loss: 0.0116 - accuracy: 0.9977
 34/375 [=>............................] - ETA: 2s - loss: 0.0134 - accuracy: 0.9968
 41/375 [==>...........................] - ETA: 2s - loss: 0.0138 - accuracy: 0.9970
 47/375 [==>...........................] - ETA: 2s - loss: 0.0148 - accuracy: 0.9963
 53/375 [===>..........................] - ETA: 2s - loss: 0.0146 - accuracy: 0.9965
 59/375 [===>..........................] - ETA: 2s - loss: 0.0138 - accuracy: 0.9967
 66/375 [====>.........................] - ETA: 2s - loss: 0.0137 - accuracy: 0.9967
 73/375 [====>.........................] - ETA: 2s - loss: 0.0137 - accuracy: 0.9969
 80/375 [=====>........................] - ETA: 2s - loss: 0.0133 - accuracy: 0.9971
 87/375 [=====>........................] - ETA: 2s - loss: 0.0130 - accuracy: 0.9970
 94/375 [======>.......................] - ETA: 2s - loss: 0.0130 - accuracy: 0.9969
101/375 [=======>......................] - ETA: 2s - loss: 0.0130 - accuracy: 0.9969
108/375 [=======>......................] - ETA: 2s - loss: 0.0135 - accuracy: 0.9967
115/375 [========>.....................] - ETA: 2s - loss: 0.0132 - accuracy: 0.9968
121/375 [========>.....................] - ETA: 2s - loss: 0.0138 - accuracy: 0.9966
126/375 [=========>....................] - ETA: 2s - loss: 0.0143 - accuracy: 0.9966
132/375 [=========>....................] - ETA: 2s - loss: 0.0145 - accuracy: 0.9965
139/375 [==========>...................] - ETA: 1s - loss: 0.0142 - accuracy: 0.9966
146/375 [==========>...................] - ETA: 1s - loss: 0.0143 - accuracy: 0.9966
153/375 [===========>..................] - ETA: 1s - loss: 0.0141 - accuracy: 0.9966
160/375 [===========>..................] - ETA: 1s - loss: 0.0142 - accuracy: 0.9966
167/375 [============>.................] - ETA: 1s - loss: 0.0144 - accuracy: 0.9965
174/375 [============>.................] - ETA: 1s - loss: 0.0145 - accuracy: 0.9965
180/375 [=============>................] - ETA: 1s - loss: 0.0150 - accuracy: 0.9962
187/375 [=============>................] - ETA: 1s - loss: 0.0148 - accuracy: 0.9963
194/375 [==============>...............] - ETA: 1s - loss: 0.0147 - accuracy: 0.9963
200/375 [===============>..............] - ETA: 1s - loss: 0.0146 - accuracy: 0.9963
207/375 [===============>..............] - ETA: 1s - loss: 0.0143 - accuracy: 0.9964
214/375 [================>.............] - ETA: 1s - loss: 0.0142 - accuracy: 0.9964
221/375 [================>.............] - ETA: 1s - loss: 0.0140 - accuracy: 0.9964
228/375 [=================>............] - ETA: 1s - loss: 0.0141 - accuracy: 0.9964
235/375 [=================>............] - ETA: 1s - loss: 0.0142 - accuracy: 0.9963
242/375 [==================>...........] - ETA: 1s - loss: 0.0143 - accuracy: 0.9963
248/375 [==================>...........] - ETA: 1s - loss: 0.0141 - accuracy: 0.9964
254/375 [===================>..........] - ETA: 0s - loss: 0.0143 - accuracy: 0.9963
261/375 [===================>..........] - ETA: 0s - loss: 0.0142 - accuracy: 0.9963
266/375 [====================>.........] - ETA: 0s - loss: 0.0142 - accuracy: 0.9962
273/375 [====================>.........] - ETA: 0s - loss: 0.0142 - accuracy: 0.9962
280/375 [=====================>........] - ETA: 0s - loss: 0.0143 - accuracy: 0.9961
287/375 [=====================>........] - ETA: 0s - loss: 0.0141 - accuracy: 0.9961
294/375 [======================>.......] - ETA: 0s - loss: 0.0141 - accuracy: 0.9962
301/375 [=======================>......] - ETA: 0s - loss: 0.0140 - accuracy: 0.9963
308/375 [=======================>......] - ETA: 0s - loss: 0.0141 - accuracy: 0.9963
314/375 [========================>.....] - ETA: 0s - loss: 0.0141 - accuracy: 0.9963
321/375 [========================>.....] - ETA: 0s - loss: 0.0140 - accuracy: 0.9963
327/375 [=========================>....] - ETA: 0s - loss: 0.0139 - accuracy: 0.9964
335/375 [=========================>....] - ETA: 0s - loss: 0.0139 - accuracy: 0.9964
342/375 [==========================>...] - ETA: 0s - loss: 0.0140 - accuracy: 0.9964
349/375 [==========================>...] - ETA: 0s - loss: 0.0139 - accuracy: 0.9964
356/375 [===========================>..] - ETA: 0s - loss: 0.0138 - accuracy: 0.9965
363/375 [============================>.] - ETA: 0s - loss: 0.0139 - accuracy: 0.9964
370/375 [============================>.] - ETA: 0s - loss: 0.0141 - accuracy: 0.9964
375/375 [==============================] - 3s 8ms/step - loss: 0.0144 - accuracy: 0.9963

375/375 [==============================] - 4s 11ms/step - loss: 0.0144 - accuracy: 0.9963 - val_loss: 0.0812 - val_accuracy: 0.9794 - lr: 1.0000e-09
Epoch 17/20

  1/375 [..............................] - ETA: 4s - loss: 0.0042 - accuracy: 1.0000
  7/375 [..............................] - ETA: 3s - loss: 0.0066 - accuracy: 0.9989
 14/375 [>.............................] - ETA: 3s - loss: 0.0101 - accuracy: 0.9978
 20/375 [>.............................] - ETA: 3s - loss: 0.0111 - accuracy: 0.9973
 26/375 [=>............................] - ETA: 2s - loss: 0.0098 - accuracy: 0.9979
 32/375 [=>............................] - ETA: 2s - loss: 0.0105 - accuracy: 0.9976
 39/375 [==>...........................] - ETA: 2s - loss: 0.0102 - accuracy: 0.9978
 43/375 [==>...........................] - ETA: 2s - loss: 0.0099 - accuracy: 0.9978
 49/375 [==>...........................] - ETA: 2s - loss: 0.0118 - accuracy: 0.9967
 55/375 [===>..........................] - ETA: 2s - loss: 0.0119 - accuracy: 0.9963
 62/375 [===>..........................] - ETA: 2s - loss: 0.0124 - accuracy: 0.9961
 69/375 [====>.........................] - ETA: 2s - loss: 0.0118 - accuracy: 0.9964
 76/375 [=====>........................] - ETA: 2s - loss: 0.0121 - accuracy: 0.9964
 83/375 [=====>........................] - ETA: 2s - loss: 0.0118 - accuracy: 0.9965
 90/375 [======>.......................] - ETA: 2s - loss: 0.0118 - accuracy: 0.9965
 97/375 [======>.......................] - ETA: 2s - loss: 0.0130 - accuracy: 0.9961
104/375 [=======>......................] - ETA: 2s - loss: 0.0134 - accuracy: 0.9960
110/375 [=======>......................] - ETA: 2s - loss: 0.0131 - accuracy: 0.9961
117/375 [========>.....................] - ETA: 2s - loss: 0.0133 - accuracy: 0.9960
123/375 [========>.....................] - ETA: 2s - loss: 0.0133 - accuracy: 0.9961
126/375 [=========>....................] - ETA: 2s - loss: 0.0136 - accuracy: 0.9960
131/375 [=========>....................] - ETA: 2s - loss: 0.0135 - accuracy: 0.9960
137/375 [=========>....................] - ETA: 2s - loss: 0.0131 - accuracy: 0.9962
143/375 [==========>...................] - ETA: 2s - loss: 0.0137 - accuracy: 0.9960
149/375 [==========>...................] - ETA: 2s - loss: 0.0146 - accuracy: 0.9959
155/375 [===========>..................] - ETA: 2s - loss: 0.0143 - accuracy: 0.9960
161/375 [===========>..................] - ETA: 1s - loss: 0.0140 - accuracy: 0.9962
167/375 [============>.................] - ETA: 1s - loss: 0.0138 - accuracy: 0.9963
174/375 [============>.................] - ETA: 1s - loss: 0.0138 - accuracy: 0.9963
180/375 [=============>................] - ETA: 1s - loss: 0.0137 - accuracy: 0.9964
187/375 [=============>................] - ETA: 1s - loss: 0.0137 - accuracy: 0.9963
194/375 [==============>...............] - ETA: 1s - loss: 0.0136 - accuracy: 0.9964
201/375 [===============>..............] - ETA: 1s - loss: 0.0139 - accuracy: 0.9964
208/375 [===============>..............] - ETA: 1s - loss: 0.0139 - accuracy: 0.9964
215/375 [================>.............] - ETA: 1s - loss: 0.0142 - accuracy: 0.9963
222/375 [================>.............] - ETA: 1s - loss: 0.0141 - accuracy: 0.9963
229/375 [=================>............] - ETA: 1s - loss: 0.0141 - accuracy: 0.9963
237/375 [=================>............] - ETA: 1s - loss: 0.0139 - accuracy: 0.9964
244/375 [==================>...........] - ETA: 1s - loss: 0.0139 - accuracy: 0.9965
251/375 [===================>..........] - ETA: 1s - loss: 0.0140 - accuracy: 0.9965
258/375 [===================>..........] - ETA: 1s - loss: 0.0139 - accuracy: 0.9965
265/375 [====================>.........] - ETA: 0s - loss: 0.0141 - accuracy: 0.9964
272/375 [====================>.........] - ETA: 0s - loss: 0.0142 - accuracy: 0.9964
278/375 [=====================>........] - ETA: 0s - loss: 0.0141 - accuracy: 0.9964
285/375 [=====================>........] - ETA: 0s - loss: 0.0143 - accuracy: 0.9964
292/375 [======================>.......] - ETA: 0s - loss: 0.0141 - accuracy: 0.9965
300/375 [=======================>......] - ETA: 0s - loss: 0.0144 - accuracy: 0.9964
307/375 [=======================>......] - ETA: 0s - loss: 0.0143 - accuracy: 0.9964
314/375 [========================>.....] - ETA: 0s - loss: 0.0142 - accuracy: 0.9964
319/375 [========================>.....] - ETA: 0s - loss: 0.0144 - accuracy: 0.9963
321/375 [========================>.....] - ETA: 0s - loss: 0.0143 - accuracy: 0.9963
327/375 [=========================>....] - ETA: 0s - loss: 0.0142 - accuracy: 0.9964
334/375 [=========================>....] - ETA: 0s - loss: 0.0142 - accuracy: 0.9964
339/375 [==========================>...] - ETA: 0s - loss: 0.0142 - accuracy: 0.9963
345/375 [==========================>...] - ETA: 0s - loss: 0.0142 - accuracy: 0.9963
352/375 [===========================>..] - ETA: 0s - loss: 0.0143 - accuracy: 0.9963
358/375 [===========================>..] - ETA: 0s - loss: 0.0145 - accuracy: 0.9962
365/375 [============================>.] - ETA: 0s - loss: 0.0144 - accuracy: 0.9963
372/375 [============================>.] - ETA: 0s - loss: 0.0145 - accuracy: 0.9963
375/375 [==============================] - 3s 9ms/step - loss: 0.0144 - accuracy: 0.9963

375/375 [==============================] - 4s 11ms/step - loss: 0.0144 - accuracy: 0.9963 - val_loss: 0.0812 - val_accuracy: 0.9794 - lr: 1.0000e-10
Epoch 18/20

  1/375 [..............................] - ETA: 2s - loss: 0.0100 - accuracy: 1.0000
  7/375 [..............................] - ETA: 3s - loss: 0.0177 - accuracy: 0.9967
 14/375 [>.............................] - ETA: 3s - loss: 0.0177 - accuracy: 0.9955
 21/375 [>.............................] - ETA: 2s - loss: 0.0150 - accuracy: 0.9963
 26/375 [=>............................] - ETA: 3s - loss: 0.0149 - accuracy: 0.9964
 32/375 [=>............................] - ETA: 3s - loss: 0.0143 - accuracy: 0.9963
 39/375 [==>...........................] - ETA: 2s - loss: 0.0132 - accuracy: 0.9966
 45/375 [==>...........................] - ETA: 2s - loss: 0.0140 - accuracy: 0.9964
 51/375 [===>..........................] - ETA: 2s - loss: 0.0146 - accuracy: 0.9963
 58/375 [===>..........................] - ETA: 2s - loss: 0.0145 - accuracy: 0.9964
 65/375 [====>.........................] - ETA: 2s - loss: 0.0147 - accuracy: 0.9962
 72/375 [====>.........................] - ETA: 2s - loss: 0.0147 - accuracy: 0.9962
 79/375 [=====>........................] - ETA: 2s - loss: 0.0141 - accuracy: 0.9965
 86/375 [=====>........................] - ETA: 2s - loss: 0.0150 - accuracy: 0.9964
 93/375 [======>.......................] - ETA: 2s - loss: 0.0150 - accuracy: 0.9964
100/375 [=======>......................] - ETA: 2s - loss: 0.0152 - accuracy: 0.9965
107/375 [=======>......................] - ETA: 2s - loss: 0.0150 - accuracy: 0.9965
114/375 [========>.....................] - ETA: 2s - loss: 0.0148 - accuracy: 0.9965
121/375 [========>.....................] - ETA: 2s - loss: 0.0151 - accuracy: 0.9964
128/375 [=========>....................] - ETA: 2s - loss: 0.0152 - accuracy: 0.9963
135/375 [=========>....................] - ETA: 1s - loss: 0.0149 - accuracy: 0.9964
142/375 [==========>...................] - ETA: 1s - loss: 0.0148 - accuracy: 0.9964
149/375 [==========>...................] - ETA: 1s - loss: 0.0148 - accuracy: 0.9964
156/375 [===========>..................] - ETA: 1s - loss: 0.0148 - accuracy: 0.9963
163/375 [============>.................] - ETA: 1s - loss: 0.0149 - accuracy: 0.9963
170/375 [============>.................] - ETA: 1s - loss: 0.0152 - accuracy: 0.9962
176/375 [=============>................] - ETA: 1s - loss: 0.0149 - accuracy: 0.9963
183/375 [=============>................] - ETA: 1s - loss: 0.0148 - accuracy: 0.9964
189/375 [==============>...............] - ETA: 1s - loss: 0.0147 - accuracy: 0.9964
196/375 [==============>...............] - ETA: 1s - loss: 0.0148 - accuracy: 0.9963
203/375 [===============>..............] - ETA: 1s - loss: 0.0147 - accuracy: 0.9963
210/375 [===============>..............] - ETA: 1s - loss: 0.0145 - accuracy: 0.9963
217/375 [================>.............] - ETA: 1s - loss: 0.0145 - accuracy: 0.9963
224/375 [================>.............] - ETA: 1s - loss: 0.0147 - accuracy: 0.9962
231/375 [=================>............] - ETA: 1s - loss: 0.0145 - accuracy: 0.9963
238/375 [==================>...........] - ETA: 1s - loss: 0.0144 - accuracy: 0.9963
245/375 [==================>...........] - ETA: 1s - loss: 0.0146 - accuracy: 0.9962
252/375 [===================>..........] - ETA: 0s - loss: 0.0146 - accuracy: 0.9963
259/375 [===================>..........] - ETA: 0s - loss: 0.0145 - accuracy: 0.9963
265/375 [====================>.........] - ETA: 0s - loss: 0.0145 - accuracy: 0.9963
272/375 [====================>.........] - ETA: 0s - loss: 0.0144 - accuracy: 0.9964
277/375 [=====================>........] - ETA: 0s - loss: 0.0145 - accuracy: 0.9963
284/375 [=====================>........] - ETA: 0s - loss: 0.0145 - accuracy: 0.9963
291/375 [======================>.......] - ETA: 0s - loss: 0.0144 - accuracy: 0.9962
298/375 [======================>.......] - ETA: 0s - loss: 0.0144 - accuracy: 0.9963
305/375 [=======================>......] - ETA: 0s - loss: 0.0142 - accuracy: 0.9963
312/375 [=======================>......] - ETA: 0s - loss: 0.0146 - accuracy: 0.9962
318/375 [========================>.....] - ETA: 0s - loss: 0.0145 - accuracy: 0.9962
325/375 [=========================>....] - ETA: 0s - loss: 0.0146 - accuracy: 0.9962
332/375 [=========================>....] - ETA: 0s - loss: 0.0146 - accuracy: 0.9962
339/375 [==========================>...] - ETA: 0s - loss: 0.0146 - accuracy: 0.9962
346/375 [==========================>...] - ETA: 0s - loss: 0.0146 - accuracy: 0.9963
353/375 [===========================>..] - ETA: 0s - loss: 0.0145 - accuracy: 0.9963
360/375 [===========================>..] - ETA: 0s - loss: 0.0144 - accuracy: 0.9963
367/375 [============================>.] - ETA: 0s - loss: 0.0145 - accuracy: 0.9963
373/375 [============================>.] - ETA: 0s - loss: 0.0144 - accuracy: 0.9963
375/375 [==============================] - 3s 8ms/step - loss: 0.0144 - accuracy: 0.9963

375/375 [==============================] - 4s 11ms/step - loss: 0.0144 - accuracy: 0.9963 - val_loss: 0.0812 - val_accuracy: 0.9794 - lr: 1.0000e-11
Epoch 19/20

  1/375 [..............................] - ETA: 4s - loss: 0.0421 - accuracy: 0.9844
  8/375 [..............................] - ETA: 3s - loss: 0.0132 - accuracy: 0.9961
 15/375 [>.............................] - ETA: 3s - loss: 0.0179 - accuracy: 0.9948
 21/375 [>.............................] - ETA: 2s - loss: 0.0145 - accuracy: 0.9959
 27/375 [=>............................] - ETA: 2s - loss: 0.0143 - accuracy: 0.9962
 33/375 [=>............................] - ETA: 2s - loss: 0.0131 - accuracy: 0.9969
 37/375 [=>............................] - ETA: 3s - loss: 0.0145 - accuracy: 0.9966
 43/375 [==>...........................] - ETA: 3s - loss: 0.0148 - accuracy: 0.9964
 50/375 [===>..........................] - ETA: 2s - loss: 0.0160 - accuracy: 0.9961
 56/375 [===>..........................] - ETA: 2s - loss: 0.0156 - accuracy: 0.9961
 63/375 [====>.........................] - ETA: 2s - loss: 0.0171 - accuracy: 0.9955
 70/375 [====>.........................] - ETA: 2s - loss: 0.0162 - accuracy: 0.9959
 77/375 [=====>........................] - ETA: 2s - loss: 0.0162 - accuracy: 0.9958
 84/375 [=====>........................] - ETA: 2s - loss: 0.0157 - accuracy: 0.9959
 91/375 [======>.......................] - ETA: 2s - loss: 0.0167 - accuracy: 0.9957
 98/375 [======>.......................] - ETA: 2s - loss: 0.0166 - accuracy: 0.9956
105/375 [=======>......................] - ETA: 2s - loss: 0.0159 - accuracy: 0.9959
113/375 [========>.....................] - ETA: 2s - loss: 0.0161 - accuracy: 0.9959
120/375 [========>.....................] - ETA: 2s - loss: 0.0160 - accuracy: 0.9959
127/375 [=========>....................] - ETA: 2s - loss: 0.0166 - accuracy: 0.9958
134/375 [=========>....................] - ETA: 1s - loss: 0.0164 - accuracy: 0.9959
141/375 [==========>...................] - ETA: 1s - loss: 0.0165 - accuracy: 0.9960
148/375 [==========>...................] - ETA: 1s - loss: 0.0162 - accuracy: 0.9961
155/375 [===========>..................] - ETA: 1s - loss: 0.0163 - accuracy: 0.9960
162/375 [===========>..................] - ETA: 1s - loss: 0.0164 - accuracy: 0.9960
167/375 [============>.................] - ETA: 1s - loss: 0.0161 - accuracy: 0.9961
174/375 [============>.................] - ETA: 1s - loss: 0.0159 - accuracy: 0.9961
181/375 [=============>................] - ETA: 1s - loss: 0.0157 - accuracy: 0.9961
187/375 [=============>................] - ETA: 1s - loss: 0.0155 - accuracy: 0.9962
194/375 [==============>...............] - ETA: 1s - loss: 0.0152 - accuracy: 0.9962
201/375 [===============>..............] - ETA: 1s - loss: 0.0152 - accuracy: 0.9962
208/375 [===============>..............] - ETA: 1s - loss: 0.0149 - accuracy: 0.9963
215/375 [================>.............] - ETA: 1s - loss: 0.0146 - accuracy: 0.9964
222/375 [================>.............] - ETA: 1s - loss: 0.0151 - accuracy: 0.9963
229/375 [=================>............] - ETA: 1s - loss: 0.0149 - accuracy: 0.9963
236/375 [=================>............] - ETA: 1s - loss: 0.0148 - accuracy: 0.9963
244/375 [==================>...........] - ETA: 1s - loss: 0.0145 - accuracy: 0.9964
251/375 [===================>..........] - ETA: 0s - loss: 0.0144 - accuracy: 0.9965
257/375 [===================>..........] - ETA: 0s - loss: 0.0144 - accuracy: 0.9964
263/375 [====================>.........] - ETA: 0s - loss: 0.0145 - accuracy: 0.9963
270/375 [====================>.........] - ETA: 0s - loss: 0.0144 - accuracy: 0.9964
277/375 [=====================>........] - ETA: 0s - loss: 0.0143 - accuracy: 0.9964
284/375 [=====================>........] - ETA: 0s - loss: 0.0142 - accuracy: 0.9964
291/375 [======================>.......] - ETA: 0s - loss: 0.0142 - accuracy: 0.9964
298/375 [======================>.......] - ETA: 0s - loss: 0.0143 - accuracy: 0.9964
305/375 [=======================>......] - ETA: 0s - loss: 0.0142 - accuracy: 0.9964
312/375 [=======================>......] - ETA: 0s - loss: 0.0143 - accuracy: 0.9964
318/375 [========================>.....] - ETA: 0s - loss: 0.0142 - accuracy: 0.9964
325/375 [=========================>....] - ETA: 0s - loss: 0.0143 - accuracy: 0.9964
332/375 [=========================>....] - ETA: 0s - loss: 0.0142 - accuracy: 0.9964
339/375 [==========================>...] - ETA: 0s - loss: 0.0141 - accuracy: 0.9964
346/375 [==========================>...] - ETA: 0s - loss: 0.0143 - accuracy: 0.9963
353/375 [===========================>..] - ETA: 0s - loss: 0.0145 - accuracy: 0.9963
359/375 [===========================>..] - ETA: 0s - loss: 0.0145 - accuracy: 0.9963
365/375 [============================>.] - ETA: 0s - loss: 0.0145 - accuracy: 0.9963
372/375 [============================>.] - ETA: 0s - loss: 0.0144 - accuracy: 0.9963
375/375 [==============================] - 3s 8ms/step - loss: 0.0144 - accuracy: 0.9963

375/375 [==============================] - 4s 10ms/step - loss: 0.0144 - accuracy: 0.9963 - val_loss: 0.0812 - val_accuracy: 0.9794 - lr: 1.0000e-12
Epoch 20/20

  1/375 [..............................] - ETA: 4s - loss: 0.0172 - accuracy: 0.9922
  8/375 [..............................] - ETA: 2s - loss: 0.0160 - accuracy: 0.9951
 15/375 [>.............................] - ETA: 2s - loss: 0.0125 - accuracy: 0.9969
 21/375 [>.............................] - ETA: 2s - loss: 0.0124 - accuracy: 0.9970
 27/375 [=>............................] - ETA: 2s - loss: 0.0117 - accuracy: 0.9977
 33/375 [=>............................] - ETA: 2s - loss: 0.0115 - accuracy: 0.9976
 39/375 [==>...........................] - ETA: 2s - loss: 0.0110 - accuracy: 0.9976
 45/375 [==>...........................] - ETA: 2s - loss: 0.0125 - accuracy: 0.9974
 51/375 [===>..........................] - ETA: 2s - loss: 0.0124 - accuracy: 0.9972
 58/375 [===>..........................] - ETA: 2s - loss: 0.0126 - accuracy: 0.9972
 65/375 [====>.........................] - ETA: 2s - loss: 0.0135 - accuracy: 0.9969
 71/375 [====>.........................] - ETA: 2s - loss: 0.0134 - accuracy: 0.9968
 78/375 [=====>........................] - ETA: 2s - loss: 0.0134 - accuracy: 0.9968
 85/375 [=====>........................] - ETA: 2s - loss: 0.0132 - accuracy: 0.9968
 92/375 [======>.......................] - ETA: 2s - loss: 0.0133 - accuracy: 0.9968
 99/375 [======>.......................] - ETA: 2s - loss: 0.0128 - accuracy: 0.9970
106/375 [=======>......................] - ETA: 2s - loss: 0.0127 - accuracy: 0.9971
113/375 [========>.....................] - ETA: 2s - loss: 0.0133 - accuracy: 0.9970
120/375 [========>.....................] - ETA: 2s - loss: 0.0129 - accuracy: 0.9971
126/375 [=========>....................] - ETA: 2s - loss: 0.0127 - accuracy: 0.9972
132/375 [=========>....................] - ETA: 2s - loss: 0.0127 - accuracy: 0.9972
139/375 [==========>...................] - ETA: 1s - loss: 0.0128 - accuracy: 0.9971
146/375 [==========>...................] - ETA: 1s - loss: 0.0129 - accuracy: 0.9971
153/375 [===========>..................] - ETA: 1s - loss: 0.0131 - accuracy: 0.9969
160/375 [===========>..................] - ETA: 1s - loss: 0.0133 - accuracy: 0.9968
167/375 [============>.................] - ETA: 1s - loss: 0.0134 - accuracy: 0.9969
174/375 [============>.................] - ETA: 1s - loss: 0.0134 - accuracy: 0.9969
181/375 [=============>................] - ETA: 1s - loss: 0.0138 - accuracy: 0.9968
188/375 [==============>...............] - ETA: 1s - loss: 0.0139 - accuracy: 0.9968
195/375 [==============>...............] - ETA: 1s - loss: 0.0138 - accuracy: 0.9967
202/375 [===============>..............] - ETA: 1s - loss: 0.0139 - accuracy: 0.9967
209/375 [===============>..............] - ETA: 1s - loss: 0.0144 - accuracy: 0.9966
216/375 [================>.............] - ETA: 1s - loss: 0.0142 - accuracy: 0.9966
223/375 [================>.............] - ETA: 1s - loss: 0.0144 - accuracy: 0.9965
230/375 [=================>............] - ETA: 1s - loss: 0.0143 - accuracy: 0.9965
237/375 [=================>............] - ETA: 1s - loss: 0.0142 - accuracy: 0.9966
244/375 [==================>...........] - ETA: 1s - loss: 0.0141 - accuracy: 0.9966
251/375 [===================>..........] - ETA: 1s - loss: 0.0142 - accuracy: 0.9966
258/375 [===================>..........] - ETA: 0s - loss: 0.0140 - accuracy: 0.9966
265/375 [====================>.........] - ETA: 0s - loss: 0.0141 - accuracy: 0.9966
272/375 [====================>.........] - ETA: 0s - loss: 0.0142 - accuracy: 0.9965
279/375 [=====================>........] - ETA: 0s - loss: 0.0142 - accuracy: 0.9965
285/375 [=====================>........] - ETA: 0s - loss: 0.0142 - accuracy: 0.9965
292/375 [======================>.......] - ETA: 0s - loss: 0.0143 - accuracy: 0.9964
299/375 [======================>.......] - ETA: 0s - loss: 0.0147 - accuracy: 0.9963
306/375 [=======================>......] - ETA: 0s - loss: 0.0147 - accuracy: 0.9962
312/375 [=======================>......] - ETA: 0s - loss: 0.0147 - accuracy: 0.9962
319/375 [========================>.....] - ETA: 0s - loss: 0.0145 - accuracy: 0.9963
326/375 [=========================>....] - ETA: 0s - loss: 0.0144 - accuracy: 0.9963
333/375 [=========================>....] - ETA: 0s - loss: 0.0144 - accuracy: 0.9963
340/375 [==========================>...] - ETA: 0s - loss: 0.0145 - accuracy: 0.9963
347/375 [==========================>...] - ETA: 0s - loss: 0.0145 - accuracy: 0.9963
354/375 [===========================>..] - ETA: 0s - loss: 0.0146 - accuracy: 0.9962
361/375 [===========================>..] - ETA: 0s - loss: 0.0145 - accuracy: 0.9963
368/375 [============================>.] - ETA: 0s - loss: 0.0144 - accuracy: 0.9963
375/375 [==============================] - 3s 8ms/step - loss: 0.0144 - accuracy: 0.9963

375/375 [==============================] - 4s 10ms/step - loss: 0.0144 - accuracy: 0.9963 - val_loss: 0.0812 - val_accuracy: 0.9794 - lr: 1.0000e-13

Smart experimenting

As we start to experiment more, it becomes harder to organize and compare our results. Let’s make it more efficient by:

• Creating a function that allows us to dynamically change the number of layers and units.
• Using callback_tensorboard() which allows us to save and visually compare results.

Tip: there is another approach to performing grid searches. See the extras notebook https://misk-data-science.github.io/misk-dl/notebooks/99x2-imdb-grid-search.nb.html for details.

train_model <- function(n_units, n_layers, log_to) {
  
  # Create a model with a single hidden input layer
  model <- keras_model_sequential() %>%
    layer_dense(units = n_units, activation = "relu", input_shape = n_feat)
  
  # Add additional hidden layers based on input
  if (n_layers > 1) {
    for (i in seq_along(n_layers - 1)) {
      model %>% layer_dense(units = n_units, activation = "relu")
    }
  }
  
  # Add final output layer
  model %>% layer_dense(units = 10, activation = "softmax")
  
  # compile model
  model %>% compile(
    loss = "categorical_crossentropy",
    optimizer = optimizer_sgd(lr = 0.1, momentum = 0.9),
    metrics = "accuracy"
  )
  
  # train model and store results with callback_tensorboard()
  history <- model %>% fit(
    train_images, train_labels,
    validation_split = 0.2,
    batch_size = 128,
    epochs = 20,
    callback = list(
      callback_early_stopping(patience = 3, restore_best_weights = TRUE, min_delta = 0.0001),
      callback_reduce_lr_on_plateau(patience = 1, factor = 0.1),
      callback_tensorboard(log_dir = log_to)
      ),
    verbose = FALSE
    )
  
  return(history)
}

Now we can create a grid for various model capacities. We include an ID for each model, which we will use to save our results within callback_tensorboard().

grid <- expand_grid(
  units = c(128, 256, 512, 1024),
  layers = c(1:6)
) %>%
  mutate(id = paste0("mlp_", layers, "_layers_", units, "_units"))
grid

Now we can loop through each model capacity combination and train our models. This will take a few minutes so this is a good time to go to the 🚽.

for (row in seq_len(nrow(grid))) {
  # get parameters
  units <- grid[[row, "units"]]
  layers <- grid[[row, "layers"]]
  file_path <- paste0("mnist/", grid[[row, "id"]])
  
  # provide status update
  cat(layers, "hidden layer(s) with", units, "neurons: ")
  
  # train model
  m <- train_model(n_units = units, n_layers = layers, log_to = file_path)
  min_loss <- min(m$metrics$val_loss, na.rm = TRUE)
  
  # update status with loss
  cat(min_loss, "\n", append = TRUE)
}

Our results suggest that larger width models tend to perform better but it is unclear if deeper models add much benefit. However, to get more clarity we can analyze the learning rates with tensorboard().

Note that callback_tensorboard() saved all the model runs in /mnist subdirectory.

tensorboard("mnist")

Your turn! (3 min)

Retrain the model with:

• 2 hidden layers
• 1024 units in each hidden layer
• early stopping after 3 epochs of no improvement
• reduce learning rate after 1 epoch of no improvement

model <- keras_model_sequential() %>%
  layer_dense(units = 1024, activation = "relu", input_shape = n_feat) %>%
  layer_dense(units = 1024, activation = "relu") %>% 
  layer_dense(units = 10, activation = "softmax")

model %>% compile(
  loss = "categorical_crossentropy",
  optimizer = optimizer_sgd(lr = 0.1, momentum = 0.9),
  metrics = "accuracy"
)

history <- model %>% fit(
  train_images, train_labels,
  validation_split = 0.2,
  batch_size = 128,
  epochs = 20,
  callback = list(
    callback_early_stopping(patience = 3, restore_best_weights = TRUE, min_delta = 0.0001),
    callback_reduce_lr_on_plateau(patience = 1, factor = 0.1)
    )
  )

Regularize Overfitting

Often, once we’ve found a model that minimizes the loss, there is still some overfitting that is occuring…sometimes a lot. So our next objective is to try flatten the validation loss learning curve and bring it as close to the training loss curve as possible.

Weight decay

A common way to mitigate overfitting is to put constraints on the complexity of a network by forcing its weights to take on small values, which makes the distribution of weight values more regular. This is called weight regularization and its done by adding to the loss function of the network a cost associated with having large weights.

If you a familiar with regularized regression ℹ️ (lasso, ridge, elastic nets) then weight regularization is essentially the same thing.

\[Loss = MSE + \lambda \sum^p_{j=1} w^2_j\]

Tips:

• Although you can use L1, L2 or a combination, L2 is by far the most common and is known as weight decay in the context of neural nets.
• Optimal values vary but when tuning we typically start with factors of \(10^{-s}\) where s ranges between 1-4 (0.1, 0.01, …, 0.0001).
• The larger the weight regularizer, the more epochs generally required to reach a minimum loss.
• Weight decay can cause a noisier learning curve so its often beneficial to increase the patience parameter for early stopping if this is noticable.

model <- keras_model_sequential() %>%
  layer_dense(
    units = 512, activation = "relu", input_shape = n_feat,
    kernel_regularizer = regularizer_l2(l = 0.001)    # regularization parameter
    ) %>%
  layer_dense(
    units = 512, activation = "relu",
    kernel_regularizer = regularizer_l2(l = 0.001)    # regularization parameter
    ) %>%
  layer_dense(units = 10, activation = "softmax")

model %>% compile(
  loss = "categorical_crossentropy",
  optimizer = optimizer_sgd(lr = 0.1, momentum = 0.9),
  metrics = "accuracy"
)

history <- model %>% fit(
  train_images, train_labels,
  validation_split = 0.2,
  batch_size = 128,
  epochs = 20,
  callback = list(
    callback_early_stopping(patience = 3, restore_best_weights = TRUE, min_delta = 0.0001),
    callback_reduce_lr_on_plateau(patience = 1, factor = 0.1)
    )
  )

history

plot(history)

Dropout

Dropout is one of the most effective and commonly used regularization techniques for neural networks. Dropout applied to a layer randomly drops out (sets to zero) a certain percentage of the output features of that layer. By randomly dropping some of a layer’s outputs we minimize the chance of fitting patterns to noise in the data, a common cause of overfitting.

Tip:

• Dropout rates typically ranges between 0.2-0.5. Sometimes higher rates are necessary but note that you will get a warning when supplying rate > 0.5.
• The higher the dropout rate, the slower the convergence so you may need to increase the number of epochs.
• Its common to apply dropout after each hidden layer and with the same rate; however, this is not necessary.

model <- keras_model_sequential() %>%
  layer_dense(units = 512, activation = "relu", input_shape = n_feat) %>%
  layer_dropout(0.3) %>%                            # regularization parameter
  layer_dense(units = 512, activation = "relu") %>%
  layer_dropout(0.3) %>%                           # regularization parameter
  layer_dense(units = 10, activation = "softmax")

model %>% compile(
  loss = "categorical_crossentropy",
  optimizer = optimizer_sgd(lr = 0.1, momentum = 0.9),
  metrics = "accuracy"
)

Warning: the `lr` argument has been renamed to `learning_rate`.

history <- model %>% fit(
  train_images, train_labels,
  validation_split = 0.2,
  batch_size = 128,
  epochs = 20,
  callback = list(
    callback_early_stopping(patience = 3, restore_best_weights = TRUE, min_delta = 0.0001),
    callback_reduce_lr_on_plateau(patience = 1, factor = 0.1)
    )
  )

Epoch 1/20

2022-08-04 16:00:59.482769: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:113] Plugin optimizer for device_type GPU is enabled.

  1/375 [..............................] - ETA: 2:38 - loss: 2.4048 - accuracy: 0.0469
  7/375 [..............................] - ETA: 3s - loss: 2.0674 - accuracy: 0.3036  
 13/375 [>.............................] - ETA: 3s - loss: 1.6122 - accuracy: 0.4675
 18/375 [>.............................] - ETA: 3s - loss: 1.3965 - accuracy: 0.5408
 24/375 [>.............................] - ETA: 3s - loss: 1.2168 - accuracy: 0.6012
 29/375 [=>............................] - ETA: 3s - loss: 1.1210 - accuracy: 0.6320
 34/375 [=>............................] - ETA: 3s - loss: 1.0352 - accuracy: 0.6631
 39/375 [==>...........................] - ETA: 3s - loss: 0.9689 - accuracy: 0.6849
 44/375 [==>...........................] - ETA: 3s - loss: 0.9134 - accuracy: 0.7028
 50/375 [===>..........................] - ETA: 3s - loss: 0.8549 - accuracy: 0.7220
 55/375 [===>..........................] - ETA: 3s - loss: 0.8086 - accuracy: 0.7376
 60/375 [===>..........................] - ETA: 3s - loss: 0.7718 - accuracy: 0.7501
 66/375 [====>.........................] - ETA: 3s - loss: 0.7365 - accuracy: 0.7625
 71/375 [====>.........................] - ETA: 3s - loss: 0.7076 - accuracy: 0.7727
 76/375 [=====>........................] - ETA: 3s - loss: 0.6814 - accuracy: 0.7812
 81/375 [=====>........................] - ETA: 2s - loss: 0.6577 - accuracy: 0.7883
 87/375 [=====>........................] - ETA: 2s - loss: 0.6342 - accuracy: 0.7957
 92/375 [======>.......................] - ETA: 2s - loss: 0.6151 - accuracy: 0.8020
 98/375 [======>.......................] - ETA: 2s - loss: 0.5972 - accuracy: 0.8084
104/375 [=======>......................] - ETA: 2s - loss: 0.5760 - accuracy: 0.8154
110/375 [=======>......................] - ETA: 2s - loss: 0.5582 - accuracy: 0.8210
115/375 [========>.....................] - ETA: 2s - loss: 0.5463 - accuracy: 0.8254
120/375 [========>.....................] - ETA: 2s - loss: 0.5329 - accuracy: 0.8298
126/375 [=========>....................] - ETA: 2s - loss: 0.5222 - accuracy: 0.8337
132/375 [=========>....................] - ETA: 2s - loss: 0.5095 - accuracy: 0.8381
138/375 [==========>...................] - ETA: 2s - loss: 0.4996 - accuracy: 0.8418
144/375 [==========>...................] - ETA: 2s - loss: 0.4886 - accuracy: 0.8456
150/375 [===========>..................] - ETA: 2s - loss: 0.4782 - accuracy: 0.8492
156/375 [===========>..................] - ETA: 2s - loss: 0.4680 - accuracy: 0.8524
162/375 [===========>..................] - ETA: 2s - loss: 0.4601 - accuracy: 0.8549
168/375 [============>.................] - ETA: 2s - loss: 0.4529 - accuracy: 0.8573
174/375 [============>.................] - ETA: 1s - loss: 0.4461 - accuracy: 0.8592
180/375 [=============>................] - ETA: 1s - loss: 0.4384 - accuracy: 0.8620
185/375 [=============>................] - ETA: 1s - loss: 0.4340 - accuracy: 0.8635
191/375 [==============>...............] - ETA: 1s - loss: 0.4264 - accuracy: 0.8659
197/375 [==============>...............] - ETA: 1s - loss: 0.4190 - accuracy: 0.8681
203/375 [===============>..............] - ETA: 1s - loss: 0.4130 - accuracy: 0.8700
209/375 [===============>..............] - ETA: 1s - loss: 0.4081 - accuracy: 0.8719
215/375 [================>.............] - ETA: 1s - loss: 0.4037 - accuracy: 0.8735
221/375 [================>.............] - ETA: 1s - loss: 0.4002 - accuracy: 0.8748
227/375 [=================>............] - ETA: 1s - loss: 0.3957 - accuracy: 0.8764
233/375 [=================>............] - ETA: 1s - loss: 0.3916 - accuracy: 0.8781
239/375 [==================>...........] - ETA: 1s - loss: 0.3874 - accuracy: 0.8796
245/375 [==================>...........] - ETA: 1s - loss: 0.3835 - accuracy: 0.8807
251/375 [===================>..........] - ETA: 1s - loss: 0.3799 - accuracy: 0.8820
258/375 [===================>..........] - ETA: 1s - loss: 0.3760 - accuracy: 0.8831
264/375 [====================>.........] - ETA: 1s - loss: 0.3722 - accuracy: 0.8844
270/375 [====================>.........] - ETA: 0s - loss: 0.3691 - accuracy: 0.8854
276/375 [=====================>........] - ETA: 0s - loss: 0.3656 - accuracy: 0.8865
282/375 [=====================>........] - ETA: 0s - loss: 0.3619 - accuracy: 0.8875
288/375 [======================>.......] - ETA: 0s - loss: 0.3579 - accuracy: 0.8888
294/375 [======================>.......] - ETA: 0s - loss: 0.3545 - accuracy: 0.8900
301/375 [=======================>......] - ETA: 0s - loss: 0.3507 - accuracy: 0.8912
307/375 [=======================>......] - ETA: 0s - loss: 0.3478 - accuracy: 0.8920
313/375 [========================>.....] - ETA: 0s - loss: 0.3444 - accuracy: 0.8930
319/375 [========================>.....] - ETA: 0s - loss: 0.3415 - accuracy: 0.8940
325/375 [=========================>....] - ETA: 0s - loss: 0.3379 - accuracy: 0.8949
329/375 [=========================>....] - ETA: 0s - loss: 0.3364 - accuracy: 0.8955
334/375 [=========================>....] - ETA: 0s - loss: 0.3343 - accuracy: 0.8963
340/375 [==========================>...] - ETA: 0s - loss: 0.3316 - accuracy: 0.8972
344/375 [==========================>...] - ETA: 0s - loss: 0.3303 - accuracy: 0.8977
350/375 [===========================>..] - ETA: 0s - loss: 0.3278 - accuracy: 0.8985
356/375 [===========================>..] - ETA: 0s - loss: 0.3255 - accuracy: 0.8992
362/375 [===========================>..] - ETA: 0s - loss: 0.3238 - accuracy: 0.8998
369/375 [============================>.] - ETA: 0s - loss: 0.3199 - accuracy: 0.9010
375/375 [==============================] - 4s 9ms/step - loss: 0.3170 - accuracy: 0.9018

2022-08-04 16:01:03.369005: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:113] Plugin optimizer for device_type GPU is enabled.


375/375 [==============================] - 5s 13ms/step - loss: 0.3170 - accuracy: 0.9018 - val_loss: 0.1470 - val_accuracy: 0.9552 - lr: 0.1000
Epoch 2/20

  1/375 [..............................] - ETA: 2s - loss: 0.1736 - accuracy: 0.9609
  7/375 [..............................] - ETA: 3s - loss: 0.1001 - accuracy: 0.9710
 11/375 [..............................] - ETA: 3s - loss: 0.1032 - accuracy: 0.9666
 17/375 [>.............................] - ETA: 3s - loss: 0.1152 - accuracy: 0.9651
 21/375 [>.............................] - ETA: 3s - loss: 0.1200 - accuracy: 0.9632
 27/375 [=>............................] - ETA: 3s - loss: 0.1235 - accuracy: 0.9633
 33/375 [=>............................] - ETA: 3s - loss: 0.1292 - accuracy: 0.9621
 39/375 [==>...........................] - ETA: 3s - loss: 0.1319 - accuracy: 0.9599
 46/375 [==>...........................] - ETA: 3s - loss: 0.1321 - accuracy: 0.9599
 52/375 [===>..........................] - ETA: 3s - loss: 0.1313 - accuracy: 0.9596
 58/375 [===>..........................] - ETA: 2s - loss: 0.1299 - accuracy: 0.9600
 64/375 [====>.........................] - ETA: 2s - loss: 0.1293 - accuracy: 0.9600
 70/375 [====>.........................] - ETA: 2s - loss: 0.1291 - accuracy: 0.9595
 76/375 [=====>........................] - ETA: 2s - loss: 0.1305 - accuracy: 0.9591
 82/375 [=====>........................] - ETA: 2s - loss: 0.1323 - accuracy: 0.9582
 88/375 [======>.......................] - ETA: 2s - loss: 0.1351 - accuracy: 0.9578
 94/375 [======>.......................] - ETA: 2s - loss: 0.1368 - accuracy: 0.9572
100/375 [=======>......................] - ETA: 2s - loss: 0.1372 - accuracy: 0.9568
106/375 [=======>......................] - ETA: 2s - loss: 0.1392 - accuracy: 0.9559
112/375 [=======>......................] - ETA: 2s - loss: 0.1417 - accuracy: 0.9552
118/375 [========>.....................] - ETA: 2s - loss: 0.1407 - accuracy: 0.9555
123/375 [========>.....................] - ETA: 2s - loss: 0.1392 - accuracy: 0.9560
125/375 [=========>....................] - ETA: 2s - loss: 0.1391 - accuracy: 0.9562
131/375 [=========>....................] - ETA: 2s - loss: 0.1398 - accuracy: 0.9559
136/375 [=========>....................] - ETA: 2s - loss: 0.1391 - accuracy: 0.9559
142/375 [==========>...................] - ETA: 2s - loss: 0.1388 - accuracy: 0.9559
148/375 [==========>...................] - ETA: 2s - loss: 0.1388 - accuracy: 0.9560
154/375 [===========>..................] - ETA: 2s - loss: 0.1379 - accuracy: 0.9561
160/375 [===========>..................] - ETA: 2s - loss: 0.1392 - accuracy: 0.9556
166/375 [============>.................] - ETA: 1s - loss: 0.1389 - accuracy: 0.9554
172/375 [============>.................] - ETA: 1s - loss: 0.1403 - accuracy: 0.9550
176/375 [=============>................] - ETA: 1s - loss: 0.1402 - accuracy: 0.9551
182/375 [=============>................] - ETA: 1s - loss: 0.1410 - accuracy: 0.9549
188/375 [==============>...............] - ETA: 1s - loss: 0.1414 - accuracy: 0.9550
194/375 [==============>...............] - ETA: 1s - loss: 0.1419 - accuracy: 0.9548
200/375 [===============>..............] - ETA: 1s - loss: 0.1422 - accuracy: 0.9548
206/375 [===============>..............] - ETA: 1s - loss: 0.1424 - accuracy: 0.9548
212/375 [===============>..............] - ETA: 1s - loss: 0.1426 - accuracy: 0.9550
218/375 [================>.............] - ETA: 1s - loss: 0.1424 - accuracy: 0.9550
224/375 [================>.............] - ETA: 1s - loss: 0.1423 - accuracy: 0.9553
230/375 [=================>............] - ETA: 1s - loss: 0.1429 - accuracy: 0.9552
236/375 [=================>............] - ETA: 1s - loss: 0.1420 - accuracy: 0.9552
242/375 [==================>...........] - ETA: 1s - loss: 0.1415 - accuracy: 0.9553
248/375 [==================>...........] - ETA: 1s - loss: 0.1410 - accuracy: 0.9555
254/375 [===================>..........] - ETA: 1s - loss: 0.1415 - accuracy: 0.9553
260/375 [===================>..........] - ETA: 1s - loss: 0.1415 - accuracy: 0.9553
266/375 [====================>.........] - ETA: 1s - loss: 0.1423 - accuracy: 0.9552
272/375 [====================>.........] - ETA: 0s - loss: 0.1428 - accuracy: 0.9552
278/375 [=====================>........] - ETA: 0s - loss: 0.1438 - accuracy: 0.9550
284/375 [=====================>........] - ETA: 0s - loss: 0.1437 - accuracy: 0.9550
290/375 [======================>.......] - ETA: 0s - loss: 0.1441 - accuracy: 0.9549
296/375 [======================>.......] - ETA: 0s - loss: 0.1434 - accuracy: 0.9551
302/375 [=======================>......] - ETA: 0s - loss: 0.1429 - accuracy: 0.9552
306/375 [=======================>......] - ETA: 0s - loss: 0.1426 - accuracy: 0.9552
312/375 [=======================>......] - ETA: 0s - loss: 0.1419 - accuracy: 0.9555
318/375 [========================>.....] - ETA: 0s - loss: 0.1422 - accuracy: 0.9555
324/375 [========================>.....] - ETA: 0s - loss: 0.1423 - accuracy: 0.9554
330/375 [=========================>....] - ETA: 0s - loss: 0.1416 - accuracy: 0.9556
336/375 [=========================>....] - ETA: 0s - loss: 0.1412 - accuracy: 0.9558
342/375 [==========================>...] - ETA: 0s - loss: 0.1420 - accuracy: 0.9558
348/375 [==========================>...] - ETA: 0s - loss: 0.1423 - accuracy: 0.9557
354/375 [===========================>..] - ETA: 0s - loss: 0.1427 - accuracy: 0.9556
360/375 [===========================>..] - ETA: 0s - loss: 0.1425 - accuracy: 0.9557
366/375 [============================>.] - ETA: 0s - loss: 0.1418 - accuracy: 0.9560
372/375 [============================>.] - ETA: 0s - loss: 0.1412 - accuracy: 0.9563
375/375 [==============================] - 4s 9ms/step - loss: 0.1413 - accuracy: 0.9563

375/375 [==============================] - 4s 12ms/step - loss: 0.1413 - accuracy: 0.9563 - val_loss: 0.1089 - val_accuracy: 0.9682 - lr: 0.1000
Epoch 3/20

  1/375 [..............................] - ETA: 4s - loss: 0.0516 - accuracy: 0.9922
  8/375 [..............................] - ETA: 3s - loss: 0.1208 - accuracy: 0.9658
 14/375 [>.............................] - ETA: 3s - loss: 0.1157 - accuracy: 0.9643
 21/375 [>.............................] - ETA: 2s - loss: 0.1088 - accuracy: 0.9658
 27/375 [=>............................] - ETA: 2s - loss: 0.1103 - accuracy: 0.9644
 33/375 [=>............................] - ETA: 2s - loss: 0.1052 - accuracy: 0.9664
 39/375 [==>...........................] - ETA: 2s - loss: 0.1048 - accuracy: 0.9671
 45/375 [==>...........................] - ETA: 2s - loss: 0.1076 - accuracy: 0.9665
 50/375 [===>..........................] - ETA: 2s - loss: 0.1069 - accuracy: 0.9677
 55/375 [===>..........................] - ETA: 2s - loss: 0.1060 - accuracy: 0.9676
 61/375 [===>..........................] - ETA: 2s - loss: 0.1061 - accuracy: 0.9667
 67/375 [====>.........................] - ETA: 2s - loss: 0.1042 - accuracy: 0.9670
 72/375 [====>.........................] - ETA: 2s - loss: 0.1033 - accuracy: 0.9671
 78/375 [=====>........................] - ETA: 2s - loss: 0.1040 - accuracy: 0.9670
 84/375 [=====>........................] - ETA: 2s - loss: 0.1055 - accuracy: 0.9664
 90/375 [======>.......................] - ETA: 2s - loss: 0.1063 - accuracy: 0.9663
 96/375 [======>.......................] - ETA: 2s - loss: 0.1080 - accuracy: 0.9653
102/375 [=======>......................] - ETA: 2s - loss: 0.1085 - accuracy: 0.9649
108/375 [=======>......................] - ETA: 2s - loss: 0.1095 - accuracy: 0.9651
114/375 [========>.....................] - ETA: 2s - loss: 0.1107 - accuracy: 0.9648
121/375 [========>.....................] - ETA: 2s - loss: 0.1115 - accuracy: 0.9644
127/375 [=========>....................] - ETA: 2s - loss: 0.1111 - accuracy: 0.9646
133/375 [=========>....................] - ETA: 2s - loss: 0.1100 - accuracy: 0.9649
139/375 [==========>...................] - ETA: 2s - loss: 0.1084 - accuracy: 0.9654
145/375 [==========>...................] - ETA: 2s - loss: 0.1082 - accuracy: 0.9654
151/375 [===========>..................] - ETA: 2s - loss: 0.1083 - accuracy: 0.9652
157/375 [===========>..................] - ETA: 1s - loss: 0.1086 - accuracy: 0.9650
162/375 [===========>..................] - ETA: 1s - loss: 0.1095 - accuracy: 0.9646
168/375 [============>.................] - ETA: 1s - loss: 0.1092 - accuracy: 0.9646
174/375 [============>.................] - ETA: 1s - loss: 0.1100 - accuracy: 0.9643
180/375 [=============>................] - ETA: 1s - loss: 0.1096 - accuracy: 0.9645
185/375 [=============>................] - ETA: 1s - loss: 0.1099 - accuracy: 0.9645
191/375 [==============>...............] - ETA: 1s - loss: 0.1090 - accuracy: 0.9648
197/375 [==============>...............] - ETA: 1s - loss: 0.1090 - accuracy: 0.9648
203/375 [===============>..............] - ETA: 1s - loss: 0.1094 - accuracy: 0.9648
209/375 [===============>..............] - ETA: 1s - loss: 0.1091 - accuracy: 0.9647
215/375 [================>.............] - ETA: 1s - loss: 0.1094 - accuracy: 0.9644
220/375 [================>.............] - ETA: 1s - loss: 0.1089 - accuracy: 0.9645
225/375 [=================>............] - ETA: 1s - loss: 0.1083 - accuracy: 0.9648
231/375 [=================>............] - ETA: 1s - loss: 0.1082 - accuracy: 0.9648
237/375 [=================>............] - ETA: 1s - loss: 0.1082 - accuracy: 0.9648
243/375 [==================>...........] - ETA: 1s - loss: 0.1090 - accuracy: 0.9645
249/375 [==================>...........] - ETA: 1s - loss: 0.1079 - accuracy: 0.9649
255/375 [===================>..........] - ETA: 1s - loss: 0.1084 - accuracy: 0.9649
261/375 [===================>..........] - ETA: 1s - loss: 0.1090 - accuracy: 0.9649
267/375 [====================>.........] - ETA: 0s - loss: 0.1089 - accuracy: 0.9649
273/375 [====================>.........] - ETA: 0s - loss: 0.1086 - accuracy: 0.9651
279/375 [=====================>........] - ETA: 0s - loss: 0.1087 - accuracy: 0.9652
285/375 [=====================>........] - ETA: 0s - loss: 0.1083 - accuracy: 0.9654
291/375 [======================>.......] - ETA: 0s - loss: 0.1079 - accuracy: 0.9657
297/375 [======================>.......] - ETA: 0s - loss: 0.1072 - accuracy: 0.9659
303/375 [=======================>......] - ETA: 0s - loss: 0.1068 - accuracy: 0.9660
309/375 [=======================>......] - ETA: 0s - loss: 0.1072 - accuracy: 0.9659
315/375 [========================>.....] - ETA: 0s - loss: 0.1079 - accuracy: 0.9658
321/375 [========================>.....] - ETA: 0s - loss: 0.1074 - accuracy: 0.9660
327/375 [=========================>....] - ETA: 0s - loss: 0.1075 - accuracy: 0.9659
332/375 [=========================>....] - ETA: 0s - loss: 0.1073 - accuracy: 0.9659
338/375 [==========================>...] - ETA: 0s - loss: 0.1076 - accuracy: 0.9659
344/375 [==========================>...] - ETA: 0s - loss: 0.1080 - accuracy: 0.9658
350/375 [===========================>..] - ETA: 0s - loss: 0.1081 - accuracy: 0.9658
355/375 [===========================>..] - ETA: 0s - loss: 0.1075 - accuracy: 0.9660
360/375 [===========================>..] - ETA: 0s - loss: 0.1071 - accuracy: 0.9661
366/375 [============================>.] - ETA: 0s - loss: 0.1071 - accuracy: 0.9662
372/375 [============================>.] - ETA: 0s - loss: 0.1069 - accuracy: 0.9664
375/375 [==============================] - 3s 9ms/step - loss: 0.1070 - accuracy: 0.9663

375/375 [==============================] - 4s 12ms/step - loss: 0.1070 - accuracy: 0.9663 - val_loss: 0.1003 - val_accuracy: 0.9679 - lr: 0.1000
Epoch 4/20

  1/375 [..............................] - ETA: 3s - loss: 0.0651 - accuracy: 0.9844
  7/375 [..............................] - ETA: 3s - loss: 0.0800 - accuracy: 0.9754
 13/375 [>.............................] - ETA: 3s - loss: 0.0874 - accuracy: 0.9742
 20/375 [>.............................] - ETA: 2s - loss: 0.0847 - accuracy: 0.9730
 26/375 [=>............................] - ETA: 2s - loss: 0.0867 - accuracy: 0.9721
 33/375 [=>............................] - ETA: 2s - loss: 0.0922 - accuracy: 0.9702
 39/375 [==>...........................] - ETA: 2s - loss: 0.0910 - accuracy: 0.9704
 45/375 [==>...........................] - ETA: 2s - loss: 0.0874 - accuracy: 0.9719
 51/375 [===>..........................] - ETA: 2s - loss: 0.0864 - accuracy: 0.9723
 57/375 [===>..........................] - ETA: 2s - loss: 0.0885 - accuracy: 0.9718
 62/375 [===>..........................] - ETA: 2s - loss: 0.0878 - accuracy: 0.9718
 68/375 [====>.........................] - ETA: 2s - loss: 0.0862 - accuracy: 0.9724
 74/375 [====>.........................] - ETA: 2s - loss: 0.0843 - accuracy: 0.9733
 81/375 [=====>........................] - ETA: 2s - loss: 0.0843 - accuracy: 0.9736
 87/375 [=====>........................] - ETA: 2s - loss: 0.0846 - accuracy: 0.9732
 93/375 [======>.......................] - ETA: 2s - loss: 0.0844 - accuracy: 0.9732
 99/375 [======>.......................] - ETA: 2s - loss: 0.0828 - accuracy: 0.9732
105/375 [=======>......................] - ETA: 2s - loss: 0.0826 - accuracy: 0.9734
111/375 [=======>......................] - ETA: 2s - loss: 0.0847 - accuracy: 0.9732
117/375 [========>.....................] - ETA: 2s - loss: 0.0859 - accuracy: 0.9727
123/375 [========>.....................] - ETA: 2s - loss: 0.0861 - accuracy: 0.9726
129/375 [=========>....................] - ETA: 2s - loss: 0.0863 - accuracy: 0.9723
135/375 [=========>....................] - ETA: 2s - loss: 0.0870 - accuracy: 0.9720
141/375 [==========>...................] - ETA: 2s - loss: 0.0875 - accuracy: 0.9720
147/375 [==========>...................] - ETA: 2s - loss: 0.0871 - accuracy: 0.9720
153/375 [===========>..................] - ETA: 1s - loss: 0.0873 - accuracy: 0.9719
159/375 [===========>..................] - ETA: 1s - loss: 0.0882 - accuracy: 0.9716
165/375 [============>.................] - ETA: 1s - loss: 0.0879 - accuracy: 0.9717
170/375 [============>.................] - ETA: 1s - loss: 0.0883 - accuracy: 0.9719
176/375 [=============>................] - ETA: 1s - loss: 0.0880 - accuracy: 0.9721
182/375 [=============>................] - ETA: 1s - loss: 0.0873 - accuracy: 0.9724
188/375 [==============>...............] - ETA: 1s - loss: 0.0872 - accuracy: 0.9722
194/375 [==============>...............] - ETA: 1s - loss: 0.0876 - accuracy: 0.9721
200/375 [===============>..............] - ETA: 1s - loss: 0.0879 - accuracy: 0.9720
206/375 [===============>..............] - ETA: 1s - loss: 0.0875 - accuracy: 0.9720
211/375 [===============>..............] - ETA: 1s - loss: 0.0886 - accuracy: 0.9717
217/375 [================>.............] - ETA: 1s - loss: 0.0892 - accuracy: 0.9716
223/375 [================>.............] - ETA: 1s - loss: 0.0900 - accuracy: 0.9714
229/375 [=================>............] - ETA: 1s - loss: 0.0903 - accuracy: 0.9715
235/375 [=================>............] - ETA: 1s - loss: 0.0909 - accuracy: 0.9714
241/375 [==================>...........] - ETA: 1s - loss: 0.0903 - accuracy: 0.9715
246/375 [==================>...........] - ETA: 1s - loss: 0.0895 - accuracy: 0.9717
250/375 [===================>..........] - ETA: 1s - loss: 0.0889 - accuracy: 0.9718
256/375 [===================>..........] - ETA: 1s - loss: 0.0884 - accuracy: 0.9720
262/375 [===================>..........] - ETA: 1s - loss: 0.0881 - accuracy: 0.9721
268/375 [====================>.........] - ETA: 0s - loss: 0.0880 - accuracy: 0.9720
274/375 [====================>.........] - ETA: 0s - loss: 0.0879 - accuracy: 0.9720
280/375 [=====================>........] - ETA: 0s - loss: 0.0877 - accuracy: 0.9722
285/375 [=====================>........] - ETA: 0s - loss: 0.0877 - accuracy: 0.9721
292/375 [======================>.......] - ETA: 0s - loss: 0.0887 - accuracy: 0.9720
298/375 [======================>.......] - ETA: 0s - loss: 0.0888 - accuracy: 0.9720
304/375 [=======================>......] - ETA: 0s - loss: 0.0890 - accuracy: 0.9719
310/375 [=======================>......] - ETA: 0s - loss: 0.0894 - accuracy: 0.9718
317/375 [========================>.....] - ETA: 0s - loss: 0.0892 - accuracy: 0.9718
323/375 [========================>.....] - ETA: 0s - loss: 0.0894 - accuracy: 0.9718
329/375 [=========================>....] - ETA: 0s - loss: 0.0893 - accuracy: 0.9719
335/375 [=========================>....] - ETA: 0s - loss: 0.0895 - accuracy: 0.9717
341/375 [==========================>...] - ETA: 0s - loss: 0.0899 - accuracy: 0.9715
347/375 [==========================>...] - ETA: 0s - loss: 0.0900 - accuracy: 0.9714
353/375 [===========================>..] - ETA: 0s - loss: 0.0902 - accuracy: 0.9715
359/375 [===========================>..] - ETA: 0s - loss: 0.0902 - accuracy: 0.9716
365/375 [============================>.] - ETA: 0s - loss: 0.0903 - accuracy: 0.9716
372/375 [============================>.] - ETA: 0s - loss: 0.0903 - accuracy: 0.9715
375/375 [==============================] - 3s 9ms/step - loss: 0.0903 - accuracy: 0.9716

375/375 [==============================] - 4s 12ms/step - loss: 0.0903 - accuracy: 0.9716 - val_loss: 0.0880 - val_accuracy: 0.9744 - lr: 0.1000
Epoch 5/20

  1/375 [..............................] - ETA: 2s - loss: 0.0071 - accuracy: 1.0000
  7/375 [..............................] - ETA: 3s - loss: 0.0633 - accuracy: 0.9754
 13/375 [>.............................] - ETA: 3s - loss: 0.0589 - accuracy: 0.9778
 19/375 [>.............................] - ETA: 3s - loss: 0.0529 - accuracy: 0.9807
 25/375 [=>............................] - ETA: 3s - loss: 0.0616 - accuracy: 0.9800
 30/375 [=>............................] - ETA: 3s - loss: 0.0575 - accuracy: 0.9818
 36/375 [=>............................] - ETA: 3s - loss: 0.0594 - accuracy: 0.9811
 38/375 [==>...........................] - ETA: 3s - loss: 0.0638 - accuracy: 0.9803
 43/375 [==>...........................] - ETA: 3s - loss: 0.0620 - accuracy: 0.9806
 49/375 [==>...........................] - ETA: 3s - loss: 0.0617 - accuracy: 0.9805
 53/375 [===>..........................] - ETA: 3s - loss: 0.0639 - accuracy: 0.9795
 59/375 [===>..........................] - ETA: 3s - loss: 0.0658 - accuracy: 0.9788
 65/375 [====>.........................] - ETA: 3s - loss: 0.0658 - accuracy: 0.9790
 71/375 [====>.........................] - ETA: 3s - loss: 0.0668 - accuracy: 0.9788
 78/375 [=====>........................] - ETA: 2s - loss: 0.0659 - accuracy: 0.9794
 84/375 [=====>........................] - ETA: 2s - loss: 0.0638 - accuracy: 0.9801
 90/375 [======>.......................] - ETA: 2s - loss: 0.0639 - accuracy: 0.9800
 96/375 [======>.......................] - ETA: 2s - loss: 0.0656 - accuracy: 0.9797
102/375 [=======>......................] - ETA: 2s - loss: 0.0641 - accuracy: 0.9802
108/375 [=======>......................] - ETA: 2s - loss: 0.0640 - accuracy: 0.9800
114/375 [========>.....................] - ETA: 2s - loss: 0.0644 - accuracy: 0.9801
120/375 [========>.....................] - ETA: 2s - loss: 0.0643 - accuracy: 0.9802
126/375 [=========>....................] - ETA: 2s - loss: 0.0653 - accuracy: 0.9797
132/375 [=========>....................] - ETA: 2s - loss: 0.0650 - accuracy: 0.9796
138/375 [==========>...................] - ETA: 2s - loss: 0.0664 - accuracy: 0.9794
144/375 [==========>...................] - ETA: 2s - loss: 0.0669 - accuracy: 0.9793
150/375 [===========>..................] - ETA: 2s - loss: 0.0671 - accuracy: 0.9790
156/375 [===========>..................] - ETA: 2s - loss: 0.0669 - accuracy: 0.9789
162/375 [===========>..................] - ETA: 2s - loss: 0.0670 - accuracy: 0.9790
168/375 [============>.................] - ETA: 1s - loss: 0.0666 - accuracy: 0.9789
174/375 [============>.................] - ETA: 1s - loss: 0.0663 - accuracy: 0.9791
180/375 [=============>................] - ETA: 1s - loss: 0.0666 - accuracy: 0.9789
186/375 [=============>................] - ETA: 1s - loss: 0.0673 - accuracy: 0.9788
192/375 [==============>...............] - ETA: 1s - loss: 0.0670 - accuracy: 0.9788
198/375 [==============>...............] - ETA: 1s - loss: 0.0667 - accuracy: 0.9789
204/375 [===============>..............] - ETA: 1s - loss: 0.0673 - accuracy: 0.9788
210/375 [===============>..............] - ETA: 1s - loss: 0.0673 - accuracy: 0.9788
216/375 [================>.............] - ETA: 1s - loss: 0.0677 - accuracy: 0.9787
222/375 [================>.............] - ETA: 1s - loss: 0.0673 - accuracy: 0.9788
228/375 [=================>............] - ETA: 1s - loss: 0.0668 - accuracy: 0.9789
234/375 [=================>............] - ETA: 1s - loss: 0.0674 - accuracy: 0.9789
240/375 [==================>...........] - ETA: 1s - loss: 0.0677 - accuracy: 0.9788
246/375 [==================>...........] - ETA: 1s - loss: 0.0679 - accuracy: 0.9787
252/375 [===================>..........] - ETA: 1s - loss: 0.0687 - accuracy: 0.9785
258/375 [===================>..........] - ETA: 1s - loss: 0.0694 - accuracy: 0.9783
264/375 [====================>.........] - ETA: 1s - loss: 0.0695 - accuracy: 0.9782
270/375 [====================>.........] - ETA: 0s - loss: 0.0703 - accuracy: 0.9781
275/375 [=====================>........] - ETA: 0s - loss: 0.0710 - accuracy: 0.9778
281/375 [=====================>........] - ETA: 0s - loss: 0.0716 - accuracy: 0.9776
287/375 [=====================>........] - ETA: 0s - loss: 0.0720 - accuracy: 0.9774
293/375 [======================>.......] - ETA: 0s - loss: 0.0722 - accuracy: 0.9773
299/375 [======================>.......] - ETA: 0s - loss: 0.0728 - accuracy: 0.9771
306/375 [=======================>......] - ETA: 0s - loss: 0.0732 - accuracy: 0.9770
312/375 [=======================>......] - ETA: 0s - loss: 0.0728 - accuracy: 0.9771
318/375 [========================>.....] - ETA: 0s - loss: 0.0726 - accuracy: 0.9772
324/375 [========================>.....] - ETA: 0s - loss: 0.0727 - accuracy: 0.9772
330/375 [=========================>....] - ETA: 0s - loss: 0.0736 - accuracy: 0.9771
336/375 [=========================>....] - ETA: 0s - loss: 0.0737 - accuracy: 0.9769
341/375 [==========================>...] - ETA: 0s - loss: 0.0738 - accuracy: 0.9769
347/375 [==========================>...] - ETA: 0s - loss: 0.0739 - accuracy: 0.9769
353/375 [===========================>..] - ETA: 0s - loss: 0.0740 - accuracy: 0.9768
359/375 [===========================>..] - ETA: 0s - loss: 0.0742 - accuracy: 0.9767
365/375 [============================>.] - ETA: 0s - loss: 0.0739 - accuracy: 0.9768
371/375 [============================>.] - ETA: 0s - loss: 0.0748 - accuracy: 0.9765
375/375 [==============================] - 4s 9ms/step - loss: 0.0748 - accuracy: 0.9765

375/375 [==============================] - 4s 12ms/step - loss: 0.0748 - accuracy: 0.9765 - val_loss: 0.0848 - val_accuracy: 0.9754 - lr: 0.1000
Epoch 6/20

  1/375 [..............................] - ETA: 3s - loss: 0.0949 - accuracy: 0.9609
  6/375 [..............................] - ETA: 4s - loss: 0.0513 - accuracy: 0.9805
 11/375 [..............................] - ETA: 4s - loss: 0.0643 - accuracy: 0.9751
 16/375 [>.............................] - ETA: 3s - loss: 0.0615 - accuracy: 0.9771
 22/375 [>.............................] - ETA: 3s - loss: 0.0591 - accuracy: 0.9787
 28/375 [=>............................] - ETA: 3s - loss: 0.0604 - accuracy: 0.9796
 34/375 [=>............................] - ETA: 3s - loss: 0.0621 - accuracy: 0.9795
 40/375 [==>...........................] - ETA: 3s - loss: 0.0611 - accuracy: 0.9805
 46/375 [==>...........................] - ETA: 3s - loss: 0.0604 - accuracy: 0.9800
 51/375 [===>..........................] - ETA: 3s - loss: 0.0613 - accuracy: 0.9799
 57/375 [===>..........................] - ETA: 3s - loss: 0.0619 - accuracy: 0.9803
 62/375 [===>..........................] - ETA: 3s - loss: 0.0602 - accuracy: 0.9807
 68/375 [====>.........................] - ETA: 3s - loss: 0.0584 - accuracy: 0.9808
 74/375 [====>.........................] - ETA: 3s - loss: 0.0563 - accuracy: 0.9812
 80/375 [=====>........................] - ETA: 2s - loss: 0.0590 - accuracy: 0.9805
 85/375 [=====>........................] - ETA: 2s - loss: 0.0593 - accuracy: 0.9805
 91/375 [======>.......................] - ETA: 2s - loss: 0.0611 - accuracy: 0.9803
 97/375 [======>.......................] - ETA: 2s - loss: 0.0607 - accuracy: 0.9804
103/375 [=======>......................] - ETA: 2s - loss: 0.0614 - accuracy: 0.9803
109/375 [=======>......................] - ETA: 2s - loss: 0.0613 - accuracy: 0.9801
115/375 [========>.....................] - ETA: 2s - loss: 0.0614 - accuracy: 0.9802
121/375 [========>.....................] - ETA: 2s - loss: 0.0628 - accuracy: 0.9798
127/375 [=========>....................] - ETA: 2s - loss: 0.0649 - accuracy: 0.9794
133/375 [=========>....................] - ETA: 2s - loss: 0.0644 - accuracy: 0.9794
139/375 [==========>...................] - ETA: 2s - loss: 0.0659 - accuracy: 0.9791
145/375 [==========>...................] - ETA: 2s - loss: 0.0661 - accuracy: 0.9791
151/375 [===========>..................] - ETA: 2s - loss: 0.0657 - accuracy: 0.9793
157/375 [===========>..................] - ETA: 2s - loss: 0.0657 - accuracy: 0.9792
163/375 [============>.................] - ETA: 2s - loss: 0.0660 - accuracy: 0.9793
169/375 [============>.................] - ETA: 1s - loss: 0.0662 - accuracy: 0.9794
175/375 [=============>................] - ETA: 1s - loss: 0.0663 - accuracy: 0.9794
181/375 [=============>................] - ETA: 1s - loss: 0.0660 - accuracy: 0.9795
187/375 [=============>................] - ETA: 1s - loss: 0.0658 - accuracy: 0.9794
193/375 [==============>...............] - ETA: 1s - loss: 0.0665 - accuracy: 0.9790
199/375 [==============>...............] - ETA: 1s - loss: 0.0660 - accuracy: 0.9790
205/375 [===============>..............] - ETA: 1s - loss: 0.0655 - accuracy: 0.9792
211/375 [===============>..............] - ETA: 1s - loss: 0.0658 - accuracy: 0.9792
217/375 [================>.............] - ETA: 1s - loss: 0.0663 - accuracy: 0.9790
221/375 [================>.............] - ETA: 1s - loss: 0.0665 - accuracy: 0.9790
227/375 [=================>............] - ETA: 1s - loss: 0.0665 - accuracy: 0.9790
232/375 [=================>............] - ETA: 1s - loss: 0.0660 - accuracy: 0.9792
238/375 [==================>...........] - ETA: 1s - loss: 0.0651 - accuracy: 0.9793
244/375 [==================>...........] - ETA: 1s - loss: 0.0654 - accuracy: 0.9791
250/375 [===================>..........] - ETA: 1s - loss: 0.0664 - accuracy: 0.9791
256/375 [===================>..........] - ETA: 1s - loss: 0.0660 - accuracy: 0.9792
262/375 [===================>..........] - ETA: 1s - loss: 0.0665 - accuracy: 0.9790
268/375 [====================>.........] - ETA: 1s - loss: 0.0665 - accuracy: 0.9790
274/375 [====================>.........] - ETA: 0s - loss: 0.0666 - accuracy: 0.9790
280/375 [=====================>........] - ETA: 0s - loss: 0.0662 - accuracy: 0.9791
286/375 [=====================>........] - ETA: 0s - loss: 0.0655 - accuracy: 0.9793
292/375 [======================>.......] - ETA: 0s - loss: 0.0652 - accuracy: 0.9794
298/375 [======================>.......] - ETA: 0s - loss: 0.0652 - accuracy: 0.9794
303/375 [=======================>......] - ETA: 0s - loss: 0.0653 - accuracy: 0.9793
309/375 [=======================>......] - ETA: 0s - loss: 0.0651 - accuracy: 0.9793
315/375 [========================>.....] - ETA: 0s - loss: 0.0650 - accuracy: 0.9792
321/375 [========================>.....] - ETA: 0s - loss: 0.0655 - accuracy: 0.9791
327/375 [=========================>....] - ETA: 0s - loss: 0.0657 - accuracy: 0.9790
333/375 [=========================>....] - ETA: 0s - loss: 0.0659 - accuracy: 0.9790
338/375 [==========================>...] - ETA: 0s - loss: 0.0661 - accuracy: 0.9789
344/375 [==========================>...] - ETA: 0s - loss: 0.0663 - accuracy: 0.9789
350/375 [===========================>..] - ETA: 0s - loss: 0.0663 - accuracy: 0.9789
356/375 [===========================>..] - ETA: 0s - loss: 0.0662 - accuracy: 0.9790
362/375 [===========================>..] - ETA: 0s - loss: 0.0660 - accuracy: 0.9789
368/375 [============================>.] - ETA: 0s - loss: 0.0658 - accuracy: 0.9790
374/375 [============================>.] - ETA: 0s - loss: 0.0656 - accuracy: 0.9789
375/375 [==============================] - 4s 9ms/step - loss: 0.0655 - accuracy: 0.9790

375/375 [==============================] - 4s 12ms/step - loss: 0.0655 - accuracy: 0.9790 - val_loss: 0.0977 - val_accuracy: 0.9741 - lr: 0.1000
Epoch 7/20

  1/375 [..............................] - ETA: 3s - loss: 0.0662 - accuracy: 0.9844
  7/375 [..............................] - ETA: 3s - loss: 0.0803 - accuracy: 0.9743
 13/375 [>.............................] - ETA: 3s - loss: 0.0689 - accuracy: 0.9772
 19/375 [>.............................] - ETA: 3s - loss: 0.0698 - accuracy: 0.9766
 25/375 [=>............................] - ETA: 3s - loss: 0.0748 - accuracy: 0.9756
 31/375 [=>............................] - ETA: 3s - loss: 0.0732 - accuracy: 0.9768
 36/375 [=>............................] - ETA: 3s - loss: 0.0719 - accuracy: 0.9774
 42/375 [==>...........................] - ETA: 3s - loss: 0.0680 - accuracy: 0.9788
 47/375 [==>...........................] - ETA: 3s - loss: 0.0652 - accuracy: 0.9804
 52/375 [===>..........................] - ETA: 3s - loss: 0.0635 - accuracy: 0.9806
 56/375 [===>..........................] - ETA: 3s - loss: 0.0617 - accuracy: 0.9812
 62/375 [===>..........................] - ETA: 3s - loss: 0.0599 - accuracy: 0.9817
 67/375 [====>.........................] - ETA: 3s - loss: 0.0592 - accuracy: 0.9818
 73/375 [====>.........................] - ETA: 2s - loss: 0.0583 - accuracy: 0.9822
 79/375 [=====>........................] - ETA: 2s - loss: 0.0569 - accuracy: 0.9826
 85/375 [=====>........................] - ETA: 2s - loss: 0.0554 - accuracy: 0.9832
 91/375 [======>.......................] - ETA: 2s - loss: 0.0547 - accuracy: 0.9832
 97/375 [======>.......................] - ETA: 2s - loss: 0.0541 - accuracy: 0.9834
104/375 [=======>......................] - ETA: 2s - loss: 0.0528 - accuracy: 0.9837
110/375 [=======>......................] - ETA: 2s - loss: 0.0539 - accuracy: 0.9832
116/375 [========>.....................] - ETA: 2s - loss: 0.0531 - accuracy: 0.9834
122/375 [========>.....................] - ETA: 2s - loss: 0.0522 - accuracy: 0.9838
128/375 [=========>....................] - ETA: 2s - loss: 0.0521 - accuracy: 0.9839
134/375 [=========>....................] - ETA: 2s - loss: 0.0517 - accuracy: 0.9839
140/375 [==========>...................] - ETA: 2s - loss: 0.0510 - accuracy: 0.9840
146/375 [==========>...................] - ETA: 2s - loss: 0.0504 - accuracy: 0.9841
152/375 [===========>..................] - ETA: 2s - loss: 0.0501 - accuracy: 0.9841
158/375 [===========>..................] - ETA: 2s - loss: 0.0502 - accuracy: 0.9841
164/375 [============>.................] - ETA: 1s - loss: 0.0505 - accuracy: 0.9839
170/375 [============>.................] - ETA: 1s - loss: 0.0507 - accuracy: 0.9839
176/375 [=============>................] - ETA: 1s - loss: 0.0499 - accuracy: 0.9841
182/375 [=============>................] - ETA: 1s - loss: 0.0504 - accuracy: 0.9840
188/375 [==============>...............] - ETA: 1s - loss: 0.0501 - accuracy: 0.9840
193/375 [==============>...............] - ETA: 1s - loss: 0.0499 - accuracy: 0.9841
199/375 [==============>...............] - ETA: 1s - loss: 0.0500 - accuracy: 0.9843
205/375 [===============>..............] - ETA: 1s - loss: 0.0495 - accuracy: 0.9843
211/375 [===============>..............] - ETA: 1s - loss: 0.0491 - accuracy: 0.9845
215/375 [================>.............] - ETA: 1s - loss: 0.0488 - accuracy: 0.9846
221/375 [================>.............] - ETA: 1s - loss: 0.0481 - accuracy: 0.9849
228/375 [=================>............] - ETA: 1s - loss: 0.0477 - accuracy: 0.9850
234/375 [=================>............] - ETA: 1s - loss: 0.0472 - accuracy: 0.9851
240/375 [==================>...........] - ETA: 1s - loss: 0.0467 - accuracy: 0.9853
246/375 [==================>...........] - ETA: 1s - loss: 0.0464 - accuracy: 0.9853
252/375 [===================>..........] - ETA: 1s - loss: 0.0463 - accuracy: 0.9853
258/375 [===================>..........] - ETA: 1s - loss: 0.0460 - accuracy: 0.9854
264/375 [====================>.........] - ETA: 1s - loss: 0.0457 - accuracy: 0.9855
270/375 [====================>.........] - ETA: 0s - loss: 0.0457 - accuracy: 0.9854
276/375 [=====================>........] - ETA: 0s - loss: 0.0457 - accuracy: 0.9855
281/375 [=====================>........] - ETA: 0s - loss: 0.0455 - accuracy: 0.9855
287/375 [=====================>........] - ETA: 0s - loss: 0.0453 - accuracy: 0.9856
293/375 [======================>.......] - ETA: 0s - loss: 0.0452 - accuracy: 0.9857
299/375 [======================>.......] - ETA: 0s - loss: 0.0447 - accuracy: 0.9859
305/375 [=======================>......] - ETA: 0s - loss: 0.0445 - accuracy: 0.9860
311/375 [=======================>......] - ETA: 0s - loss: 0.0440 - accuracy: 0.9861
317/375 [========================>.....] - ETA: 0s - loss: 0.0441 - accuracy: 0.9860
323/375 [========================>.....] - ETA: 0s - loss: 0.0439 - accuracy: 0.9860
329/375 [=========================>....] - ETA: 0s - loss: 0.0436 - accuracy: 0.9861
335/375 [=========================>....] - ETA: 0s - loss: 0.0434 - accuracy: 0.9862
341/375 [==========================>...] - ETA: 0s - loss: 0.0432 - accuracy: 0.9863
347/375 [==========================>...] - ETA: 0s - loss: 0.0431 - accuracy: 0.9863
353/375 [===========================>..] - ETA: 0s - loss: 0.0430 - accuracy: 0.9864
359/375 [===========================>..] - ETA: 0s - loss: 0.0431 - accuracy: 0.9864
365/375 [============================>.] - ETA: 0s - loss: 0.0430 - accuracy: 0.9864
370/375 [============================>.] - ETA: 0s - loss: 0.0428 - accuracy: 0.9865
375/375 [==============================] - 4s 9ms/step - loss: 0.0429 - accuracy: 0.9864

375/375 [==============================] - 4s 12ms/step - loss: 0.0429 - accuracy: 0.9864 - val_loss: 0.0690 - val_accuracy: 0.9811 - lr: 0.0100
Epoch 8/20

  1/375 [..............................] - ETA: 4s - loss: 0.0235 - accuracy: 1.0000
  7/375 [..............................] - ETA: 3s - loss: 0.0249 - accuracy: 0.9933
 13/375 [>.............................] - ETA: 3s - loss: 0.0279 - accuracy: 0.9928
 18/375 [>.............................] - ETA: 3s - loss: 0.0367 - accuracy: 0.9896
 22/375 [>.............................] - ETA: 3s - loss: 0.0378 - accuracy: 0.9890
 28/375 [=>............................] - ETA: 3s - loss: 0.0356 - accuracy: 0.9897
 34/375 [=>............................] - ETA: 3s - loss: 0.0328 - accuracy: 0.9910
 40/375 [==>...........................] - ETA: 3s - loss: 0.0342 - accuracy: 0.9906
 46/375 [==>...........................] - ETA: 3s - loss: 0.0327 - accuracy: 0.9907
 52/375 [===>..........................] - ETA: 3s - loss: 0.0337 - accuracy: 0.9907
 58/375 [===>..........................] - ETA: 3s - loss: 0.0330 - accuracy: 0.9906
 65/375 [====>.........................] - ETA: 3s - loss: 0.0330 - accuracy: 0.9904
 71/375 [====>.........................] - ETA: 2s - loss: 0.0331 - accuracy: 0.9903
 77/375 [=====>........................] - ETA: 2s - loss: 0.0331 - accuracy: 0.9900
 83/375 [=====>........................] - ETA: 2s - loss: 0.0326 - accuracy: 0.9904
 89/375 [======>.......................] - ETA: 2s - loss: 0.0318 - accuracy: 0.9904
 95/375 [======>.......................] - ETA: 2s - loss: 0.0307 - accuracy: 0.9908
101/375 [=======>......................] - ETA: 2s - loss: 0.0305 - accuracy: 0.9909
107/375 [=======>......................] - ETA: 2s - loss: 0.0309 - accuracy: 0.9908
113/375 [========>.....................] - ETA: 2s - loss: 0.0302 - accuracy: 0.9909
118/375 [========>.....................] - ETA: 2s - loss: 0.0304 - accuracy: 0.9908
124/375 [========>.....................] - ETA: 2s - loss: 0.0302 - accuracy: 0.9909
129/375 [=========>....................] - ETA: 2s - loss: 0.0311 - accuracy: 0.9906
135/375 [=========>....................] - ETA: 2s - loss: 0.0305 - accuracy: 0.9909
141/375 [==========>...................] - ETA: 2s - loss: 0.0306 - accuracy: 0.9907
146/375 [==========>...................] - ETA: 2s - loss: 0.0306 - accuracy: 0.9907
151/375 [===========>..................] - ETA: 2s - loss: 0.0307 - accuracy: 0.9907
157/375 [===========>..................] - ETA: 2s - loss: 0.0307 - accuracy: 0.9906
163/375 [============>.................] - ETA: 2s - loss: 0.0306 - accuracy: 0.9906
168/375 [============>.................] - ETA: 2s - loss: 0.0306 - accuracy: 0.9906
171/375 [============>.................] - ETA: 2s - loss: 0.0311 - accuracy: 0.9905
176/375 [=============>................] - ETA: 1s - loss: 0.0311 - accuracy: 0.9905
182/375 [=============>................] - ETA: 1s - loss: 0.0310 - accuracy: 0.9906
188/375 [==============>...............] - ETA: 1s - loss: 0.0310 - accuracy: 0.9906
193/375 [==============>...............] - ETA: 1s - loss: 0.0310 - accuracy: 0.9906
198/375 [==============>...............] - ETA: 1s - loss: 0.0308 - accuracy: 0.9906
201/375 [===============>..............] - ETA: 1s - loss: 0.0309 - accuracy: 0.9906
202/375 [===============>..............] - ETA: 1s - loss: 0.0310 - accuracy: 0.9906
205/375 [===============>..............] - ETA: 1s - loss: 0.0315 - accuracy: 0.9905
206/375 [===============>..............] - ETA: 1s - loss: 0.0315 - accuracy: 0.9906
209/375 [===============>..............] - ETA: 1s - loss: 0.0312 - accuracy: 0.9907
211/375 [===============>..............] - ETA: 1s - loss: 0.0310 - accuracy: 0.9907
214/375 [================>.............] - ETA: 1s - loss: 0.0311 - accuracy: 0.9907
216/375 [================>.............] - ETA: 1s - loss: 0.0311 - accuracy: 0.9907
219/375 [================>.............] - ETA: 1s - loss: 0.0311 - accuracy: 0.9907
220/375 [================>.............] - ETA: 1s - loss: 0.0309 - accuracy: 0.9907
222/375 [================>.............] - ETA: 1s - loss: 0.0308 - accuracy: 0.9907
223/375 [================>.............] - ETA: 1s - loss: 0.0308 - accuracy: 0.9907
225/375 [=================>............] - ETA: 1s - loss: 0.0309 - accuracy: 0.9906
226/375 [=================>............] - ETA: 1s - loss: 0.0309 - accuracy: 0.9906
228/375 [=================>............] - ETA: 1s - loss: 0.0310 - accuracy: 0.9905
230/375 [=================>............] - ETA: 1s - loss: 0.0310 - accuracy: 0.9905
236/375 [=================>............] - ETA: 1s - loss: 0.0310 - accuracy: 0.9905
242/375 [==================>...........] - ETA: 1s - loss: 0.0307 - accuracy: 0.9905
248/375 [==================>...........] - ETA: 1s - loss: 0.0309 - accuracy: 0.9905
254/375 [===================>..........] - ETA: 1s - loss: 0.0310 - accuracy: 0.9904
261/375 [===================>..........] - ETA: 1s - loss: 0.0308 - accuracy: 0.9904
267/375 [====================>.........] - ETA: 1s - loss: 0.0305 - accuracy: 0.9905
273/375 [====================>.........] - ETA: 1s - loss: 0.0305 - accuracy: 0.9905
279/375 [=====================>........] - ETA: 1s - loss: 0.0304 - accuracy: 0.9905
285/375 [=====================>........] - ETA: 1s - loss: 0.0307 - accuracy: 0.9904
291/375 [======================>.......] - ETA: 1s - loss: 0.0307 - accuracy: 0.9904
297/375 [======================>.......] - ETA: 0s - loss: 0.0305 - accuracy: 0.9906
303/375 [=======================>......] - ETA: 0s - loss: 0.0302 - accuracy: 0.9906
308/375 [=======================>......] - ETA: 0s - loss: 0.0304 - accuracy: 0.9907
314/375 [========================>.....] - ETA: 0s - loss: 0.0309 - accuracy: 0.9904
319/375 [========================>.....] - ETA: 0s - loss: 0.0311 - accuracy: 0.9904
324/375 [========================>.....] - ETA: 0s - loss: 0.0308 - accuracy: 0.9905
330/375 [=========================>....] - ETA: 0s - loss: 0.0310 - accuracy: 0.9904
335/375 [=========================>....] - ETA: 0s - loss: 0.0309 - accuracy: 0.9904
341/375 [==========================>...] - ETA: 0s - loss: 0.0308 - accuracy: 0.9904
347/375 [==========================>...] - ETA: 0s - loss: 0.0311 - accuracy: 0.9903
353/375 [===========================>..] - ETA: 0s - loss: 0.0310 - accuracy: 0.9904
359/375 [===========================>..] - ETA: 0s - loss: 0.0309 - accuracy: 0.9904
365/375 [============================>.] - ETA: 0s - loss: 0.0309 - accuracy: 0.9903
372/375 [============================>.] - ETA: 0s - loss: 0.0309 - accuracy: 0.9902
375/375 [==============================] - 4s 12ms/step - loss: 0.0307 - accuracy: 0.9903

375/375 [==============================] - 5s 14ms/step - loss: 0.0307 - accuracy: 0.9903 - val_loss: 0.0667 - val_accuracy: 0.9820 - lr: 0.0100
Epoch 9/20

  1/375 [..............................] - ETA: 3s - loss: 0.0128 - accuracy: 1.0000
  7/375 [..............................] - ETA: 3s - loss: 0.0415 - accuracy: 0.9866
 13/375 [>.............................] - ETA: 3s - loss: 0.0328 - accuracy: 0.9892
 19/375 [>.............................] - ETA: 3s - loss: 0.0329 - accuracy: 0.9889
 25/375 [=>............................] - ETA: 3s - loss: 0.0310 - accuracy: 0.9887
 31/375 [=>............................] - ETA: 3s - loss: 0.0313 - accuracy: 0.9894
 37/375 [=>............................] - ETA: 3s - loss: 0.0305 - accuracy: 0.9894
 43/375 [==>...........................] - ETA: 2s - loss: 0.0311 - accuracy: 0.9891
 49/375 [==>...........................] - ETA: 2s - loss: 0.0320 - accuracy: 0.9892
 55/375 [===>..........................] - ETA: 2s - loss: 0.0315 - accuracy: 0.9891
 61/375 [===>..........................] - ETA: 2s - loss: 0.0317 - accuracy: 0.9894
 67/375 [====>.........................] - ETA: 2s - loss: 0.0315 - accuracy: 0.9895
 73/375 [====>.........................] - ETA: 2s - loss: 0.0331 - accuracy: 0.9893
 79/375 [=====>........................] - ETA: 2s - loss: 0.0330 - accuracy: 0.9895
 86/375 [=====>........................] - ETA: 2s - loss: 0.0338 - accuracy: 0.9891
 93/375 [======>.......................] - ETA: 2s - loss: 0.0334 - accuracy: 0.9892
 99/375 [======>.......................] - ETA: 2s - loss: 0.0335 - accuracy: 0.9892
105/375 [=======>......................] - ETA: 2s - loss: 0.0333 - accuracy: 0.9891
112/375 [=======>......................] - ETA: 2s - loss: 0.0328 - accuracy: 0.9894
119/375 [========>.....................] - ETA: 2s - loss: 0.0329 - accuracy: 0.9895
125/375 [=========>....................] - ETA: 2s - loss: 0.0330 - accuracy: 0.9893
131/375 [=========>....................] - ETA: 2s - loss: 0.0326 - accuracy: 0.9894
137/375 [=========>....................] - ETA: 2s - loss: 0.0322 - accuracy: 0.9895
143/375 [==========>...................] - ETA: 2s - loss: 0.0320 - accuracy: 0.9896
149/375 [==========>...................] - ETA: 2s - loss: 0.0315 - accuracy: 0.9897
156/375 [===========>..................] - ETA: 1s - loss: 0.0309 - accuracy: 0.9899
162/375 [===========>..................] - ETA: 1s - loss: 0.0308 - accuracy: 0.9900
168/375 [============>.................] - ETA: 1s - loss: 0.0305 - accuracy: 0.9901
174/375 [============>.................] - ETA: 1s - loss: 0.0303 - accuracy: 0.9901
179/375 [=============>................] - ETA: 1s - loss: 0.0302 - accuracy: 0.9902
185/375 [=============>................] - ETA: 1s - loss: 0.0301 - accuracy: 0.9901
191/375 [==============>...............] - ETA: 1s - loss: 0.0298 - accuracy: 0.9902
196/375 [==============>...............] - ETA: 1s - loss: 0.0298 - accuracy: 0.9901
202/375 [===============>..............] - ETA: 1s - loss: 0.0293 - accuracy: 0.9903
208/375 [===============>..............] - ETA: 1s - loss: 0.0295 - accuracy: 0.9903
214/375 [================>.............] - ETA: 1s - loss: 0.0294 - accuracy: 0.9903
220/375 [================>.............] - ETA: 1s - loss: 0.0293 - accuracy: 0.9905
226/375 [=================>............] - ETA: 1s - loss: 0.0291 - accuracy: 0.9906
232/375 [=================>............] - ETA: 1s - loss: 0.0290 - accuracy: 0.9906
238/375 [==================>...........] - ETA: 1s - loss: 0.0291 - accuracy: 0.9905
244/375 [==================>...........] - ETA: 1s - loss: 0.0292 - accuracy: 0.9904
250/375 [===================>..........] - ETA: 1s - loss: 0.0288 - accuracy: 0.9906
256/375 [===================>..........] - ETA: 1s - loss: 0.0285 - accuracy: 0.9907
263/375 [====================>.........] - ETA: 1s - loss: 0.0282 - accuracy: 0.9909
269/375 [====================>.........] - ETA: 0s - loss: 0.0283 - accuracy: 0.9907
275/375 [=====================>........] - ETA: 0s - loss: 0.0282 - accuracy: 0.9907
282/375 [=====================>........] - ETA: 0s - loss: 0.0282 - accuracy: 0.9907
288/375 [======================>.......] - ETA: 0s - loss: 0.0280 - accuracy: 0.9909
295/375 [======================>.......] - ETA: 0s - loss: 0.0278 - accuracy: 0.9909
302/375 [=======================>......] - ETA: 0s - loss: 0.0277 - accuracy: 0.9910
308/375 [=======================>......] - ETA: 0s - loss: 0.0277 - accuracy: 0.9910
314/375 [========================>.....] - ETA: 0s - loss: 0.0278 - accuracy: 0.9909
320/375 [========================>.....] - ETA: 0s - loss: 0.0279 - accuracy: 0.9909
326/375 [=========================>....] - ETA: 0s - loss: 0.0279 - accuracy: 0.9909
332/375 [=========================>....] - ETA: 0s - loss: 0.0278 - accuracy: 0.9910
338/375 [==========================>...] - ETA: 0s - loss: 0.0281 - accuracy: 0.9909
343/375 [==========================>...] - ETA: 0s - loss: 0.0282 - accuracy: 0.9910
349/375 [==========================>...] - ETA: 0s - loss: 0.0280 - accuracy: 0.9910
355/375 [===========================>..] - ETA: 0s - loss: 0.0281 - accuracy: 0.9910
361/375 [===========================>..] - ETA: 0s - loss: 0.0282 - accuracy: 0.9909
366/375 [============================>.] - ETA: 0s - loss: 0.0282 - accuracy: 0.9909
372/375 [============================>.] - ETA: 0s - loss: 0.0281 - accuracy: 0.9910
375/375 [==============================] - 3s 9ms/step - loss: 0.0280 - accuracy: 0.9910

375/375 [==============================] - 4s 12ms/step - loss: 0.0280 - accuracy: 0.9910 - val_loss: 0.0644 - val_accuracy: 0.9828 - lr: 0.0100
Epoch 10/20

  1/375 [..............................] - ETA: 2s - loss: 0.0122 - accuracy: 1.0000
  7/375 [..............................] - ETA: 3s - loss: 0.0208 - accuracy: 0.9922
 13/375 [>.............................] - ETA: 3s - loss: 0.0257 - accuracy: 0.9916
 19/375 [>.............................] - ETA: 3s - loss: 0.0271 - accuracy: 0.9914
 25/375 [=>............................] - ETA: 3s - loss: 0.0266 - accuracy: 0.9912
 31/375 [=>............................] - ETA: 3s - loss: 0.0263 - accuracy: 0.9909
 37/375 [=>............................] - ETA: 3s - loss: 0.0274 - accuracy: 0.9903
 41/375 [==>...........................] - ETA: 3s - loss: 0.0271 - accuracy: 0.9905
 47/375 [==>...........................] - ETA: 3s - loss: 0.0266 - accuracy: 0.9904
 53/375 [===>..........................] - ETA: 3s - loss: 0.0264 - accuracy: 0.9906
 59/375 [===>..........................] - ETA: 3s - loss: 0.0253 - accuracy: 0.9913
 65/375 [====>.........................] - ETA: 3s - loss: 0.0252 - accuracy: 0.9911
 71/375 [====>.........................] - ETA: 2s - loss: 0.0247 - accuracy: 0.9916
 77/375 [=====>........................] - ETA: 2s - loss: 0.0246 - accuracy: 0.9917
 83/375 [=====>........................] - ETA: 2s - loss: 0.0252 - accuracy: 0.9913
 89/375 [======>.......................] - ETA: 2s - loss: 0.0264 - accuracy: 0.9910
 95/375 [======>.......................] - ETA: 2s - loss: 0.0265 - accuracy: 0.9910
101/375 [=======>......................] - ETA: 2s - loss: 0.0265 - accuracy: 0.9911
107/375 [=======>......................] - ETA: 2s - loss: 0.0260 - accuracy: 0.9911
113/375 [========>.....................] - ETA: 2s - loss: 0.0259 - accuracy: 0.9912
119/375 [========>.....................] - ETA: 2s - loss: 0.0253 - accuracy: 0.9913
125/375 [=========>....................] - ETA: 2s - loss: 0.0254 - accuracy: 0.9913
132/375 [=========>....................] - ETA: 2s - loss: 0.0249 - accuracy: 0.9915
138/375 [==========>...................] - ETA: 2s - loss: 0.0247 - accuracy: 0.9916
144/375 [==========>...................] - ETA: 2s - loss: 0.0250 - accuracy: 0.9915
150/375 [===========>..................] - ETA: 2s - loss: 0.0249 - accuracy: 0.9916
156/375 [===========>..................] - ETA: 2s - loss: 0.0254 - accuracy: 0.9917
162/375 [===========>..................] - ETA: 1s - loss: 0.0260 - accuracy: 0.9916
168/375 [============>.................] - ETA: 1s - loss: 0.0257 - accuracy: 0.9918
174/375 [============>.................] - ETA: 1s - loss: 0.0256 - accuracy: 0.9918
180/375 [=============>................] - ETA: 1s - loss: 0.0257 - accuracy: 0.9918
185/375 [=============>................] - ETA: 1s - loss: 0.0257 - accuracy: 0.9918
191/375 [==============>...............] - ETA: 1s - loss: 0.0256 - accuracy: 0.9918
197/375 [==============>...............] - ETA: 1s - loss: 0.0258 - accuracy: 0.9918
202/375 [===============>..............] - ETA: 1s - loss: 0.0260 - accuracy: 0.9918
208/375 [===============>..............] - ETA: 1s - loss: 0.0258 - accuracy: 0.9918
214/375 [================>.............] - ETA: 1s - loss: 0.0257 - accuracy: 0.9918
219/375 [================>.............] - ETA: 1s - loss: 0.0256 - accuracy: 0.9919
225/375 [=================>............] - ETA: 1s - loss: 0.0257 - accuracy: 0.9918
231/375 [=================>............] - ETA: 1s - loss: 0.0257 - accuracy: 0.9919
236/375 [=================>............] - ETA: 1s - loss: 0.0259 - accuracy: 0.9919
242/375 [==================>...........] - ETA: 1s - loss: 0.0256 - accuracy: 0.9920
248/375 [==================>...........] - ETA: 1s - loss: 0.0256 - accuracy: 0.9920
253/375 [===================>..........] - ETA: 1s - loss: 0.0253 - accuracy: 0.9921
259/375 [===================>..........] - ETA: 1s - loss: 0.0253 - accuracy: 0.9921
265/375 [====================>.........] - ETA: 1s - loss: 0.0251 - accuracy: 0.9921
270/375 [====================>.........] - ETA: 0s - loss: 0.0251 - accuracy: 0.9922
276/375 [=====================>........] - ETA: 0s - loss: 0.0250 - accuracy: 0.9921
282/375 [=====================>........] - ETA: 0s - loss: 0.0250 - accuracy: 0.9921
287/375 [=====================>........] - ETA: 0s - loss: 0.0249 - accuracy: 0.9922
293/375 [======================>.......] - ETA: 0s - loss: 0.0248 - accuracy: 0.9923
299/375 [======================>.......] - ETA: 0s - loss: 0.0248 - accuracy: 0.9922
306/375 [=======================>......] - ETA: 0s - loss: 0.0247 - accuracy: 0.9923
312/375 [=======================>......] - ETA: 0s - loss: 0.0247 - accuracy: 0.9922
318/375 [========================>.....] - ETA: 0s - loss: 0.0248 - accuracy: 0.9922
324/375 [========================>.....] - ETA: 0s - loss: 0.0247 - accuracy: 0.9923
330/375 [=========================>....] - ETA: 0s - loss: 0.0248 - accuracy: 0.9923
336/375 [=========================>....] - ETA: 0s - loss: 0.0250 - accuracy: 0.9923
342/375 [==========================>...] - ETA: 0s - loss: 0.0251 - accuracy: 0.9922
348/375 [==========================>...] - ETA: 0s - loss: 0.0251 - accuracy: 0.9923
354/375 [===========================>..] - ETA: 0s - loss: 0.0254 - accuracy: 0.9922
360/375 [===========================>..] - ETA: 0s - loss: 0.0254 - accuracy: 0.9921
366/375 [============================>.] - ETA: 0s - loss: 0.0254 - accuracy: 0.9922
372/375 [============================>.] - ETA: 0s - loss: 0.0255 - accuracy: 0.9921
375/375 [==============================] - 4s 9ms/step - loss: 0.0254 - accuracy: 0.9921

375/375 [==============================] - 4s 12ms/step - loss: 0.0254 - accuracy: 0.9921 - val_loss: 0.0639 - val_accuracy: 0.9828 - lr: 0.0100
Epoch 11/20

  1/375 [..............................] - ETA: 4s - loss: 0.0085 - accuracy: 1.0000
  7/375 [..............................] - ETA: 3s - loss: 0.0235 - accuracy: 0.9911
 13/375 [>.............................] - ETA: 3s - loss: 0.0203 - accuracy: 0.9934
 20/375 [>.............................] - ETA: 3s - loss: 0.0237 - accuracy: 0.9922
 26/375 [=>............................] - ETA: 2s - loss: 0.0229 - accuracy: 0.9925
 31/375 [=>............................] - ETA: 3s - loss: 0.0228 - accuracy: 0.9927
 37/375 [=>............................] - ETA: 3s - loss: 0.0228 - accuracy: 0.9924
 43/375 [==>...........................] - ETA: 2s - loss: 0.0211 - accuracy: 0.9935
 50/375 [===>..........................] - ETA: 2s - loss: 0.0218 - accuracy: 0.9933
 56/375 [===>..........................] - ETA: 2s - loss: 0.0218 - accuracy: 0.9932
 62/375 [===>..........................] - ETA: 2s - loss: 0.0226 - accuracy: 0.9929
 68/375 [====>.........................] - ETA: 2s - loss: 0.0237 - accuracy: 0.9922
 75/375 [=====>........................] - ETA: 2s - loss: 0.0240 - accuracy: 0.9920
 81/375 [=====>........................] - ETA: 2s - loss: 0.0253 - accuracy: 0.9917
 87/375 [=====>........................] - ETA: 2s - loss: 0.0252 - accuracy: 0.9918
 93/375 [======>.......................] - ETA: 2s - loss: 0.0255 - accuracy: 0.9918
 99/375 [======>.......................] - ETA: 2s - loss: 0.0254 - accuracy: 0.9917
105/375 [=======>......................] - ETA: 2s - loss: 0.0250 - accuracy: 0.9918
111/375 [=======>......................] - ETA: 2s - loss: 0.0252 - accuracy: 0.9918
117/375 [========>.....................] - ETA: 2s - loss: 0.0247 - accuracy: 0.9919
123/375 [========>.....................] - ETA: 2s - loss: 0.0242 - accuracy: 0.9921
129/375 [=========>....................] - ETA: 2s - loss: 0.0239 - accuracy: 0.9922
135/375 [=========>....................] - ETA: 2s - loss: 0.0238 - accuracy: 0.9923
139/375 [==========>...................] - ETA: 2s - loss: 0.0241 - accuracy: 0.9922
145/375 [==========>...................] - ETA: 2s - loss: 0.0236 - accuracy: 0.9923
151/375 [===========>..................] - ETA: 2s - loss: 0.0232 - accuracy: 0.9925
157/375 [===========>..................] - ETA: 1s - loss: 0.0231 - accuracy: 0.9926
164/375 [============>.................] - ETA: 1s - loss: 0.0229 - accuracy: 0.9928
170/375 [============>.................] - ETA: 1s - loss: 0.0228 - accuracy: 0.9928
176/375 [=============>................] - ETA: 1s - loss: 0.0226 - accuracy: 0.9928
182/375 [=============>................] - ETA: 1s - loss: 0.0229 - accuracy: 0.9927
188/375 [==============>...............] - ETA: 1s - loss: 0.0229 - accuracy: 0.9927
194/375 [==============>...............] - ETA: 1s - loss: 0.0227 - accuracy: 0.9928
200/375 [===============>..............] - ETA: 1s - loss: 0.0231 - accuracy: 0.9927
206/375 [===============>..............] - ETA: 1s - loss: 0.0235 - accuracy: 0.9925
211/375 [===============>..............] - ETA: 1s - loss: 0.0238 - accuracy: 0.9924
217/375 [================>.............] - ETA: 1s - loss: 0.0235 - accuracy: 0.9926
222/375 [================>.............] - ETA: 1s - loss: 0.0235 - accuracy: 0.9926
228/375 [=================>............] - ETA: 1s - loss: 0.0231 - accuracy: 0.9927
234/375 [=================>............] - ETA: 1s - loss: 0.0234 - accuracy: 0.9926
240/375 [==================>...........] - ETA: 1s - loss: 0.0233 - accuracy: 0.9926
246/375 [==================>...........] - ETA: 1s - loss: 0.0232 - accuracy: 0.9926
252/375 [===================>..........] - ETA: 1s - loss: 0.0236 - accuracy: 0.9925
258/375 [===================>..........] - ETA: 1s - loss: 0.0236 - accuracy: 0.9925
265/375 [====================>.........] - ETA: 0s - loss: 0.0235 - accuracy: 0.9925
271/375 [====================>.........] - ETA: 0s - loss: 0.0234 - accuracy: 0.9926
278/375 [=====================>........] - ETA: 0s - loss: 0.0232 - accuracy: 0.9926
284/375 [=====================>........] - ETA: 0s - loss: 0.0231 - accuracy: 0.9926
290/375 [======================>.......] - ETA: 0s - loss: 0.0233 - accuracy: 0.9925
297/375 [======================>.......] - ETA: 0s - loss: 0.0231 - accuracy: 0.9925
303/375 [=======================>......] - ETA: 0s - loss: 0.0230 - accuracy: 0.9926
310/375 [=======================>......] - ETA: 0s - loss: 0.0230 - accuracy: 0.9926
316/375 [========================>.....] - ETA: 0s - loss: 0.0230 - accuracy: 0.9926
323/375 [========================>.....] - ETA: 0s - loss: 0.0231 - accuracy: 0.9925
329/375 [=========================>....] - ETA: 0s - loss: 0.0232 - accuracy: 0.9924
335/375 [=========================>....] - ETA: 0s - loss: 0.0232 - accuracy: 0.9924
341/375 [==========================>...] - ETA: 0s - loss: 0.0233 - accuracy: 0.9924
348/375 [==========================>...] - ETA: 0s - loss: 0.0232 - accuracy: 0.9924
354/375 [===========================>..] - ETA: 0s - loss: 0.0230 - accuracy: 0.9925
360/375 [===========================>..] - ETA: 0s - loss: 0.0229 - accuracy: 0.9925
366/375 [============================>.] - ETA: 0s - loss: 0.0229 - accuracy: 0.9925
373/375 [============================>.] - ETA: 0s - loss: 0.0230 - accuracy: 0.9924
375/375 [==============================] - 3s 9ms/step - loss: 0.0230 - accuracy: 0.9925

375/375 [==============================] - 4s 11ms/step - loss: 0.0230 - accuracy: 0.9925 - val_loss: 0.0642 - val_accuracy: 0.9823 - lr: 0.0100
Epoch 12/20

  1/375 [..............................] - ETA: 2s - loss: 0.0044 - accuracy: 1.0000
  7/375 [..............................] - ETA: 3s - loss: 0.0293 - accuracy: 0.9922
 13/375 [>.............................] - ETA: 3s - loss: 0.0232 - accuracy: 0.9952
 20/375 [>.............................] - ETA: 2s - loss: 0.0254 - accuracy: 0.9941
 26/375 [=>............................] - ETA: 2s - loss: 0.0247 - accuracy: 0.9940
 33/375 [=>............................] - ETA: 2s - loss: 0.0242 - accuracy: 0.9934
 39/375 [==>...........................] - ETA: 2s - loss: 0.0231 - accuracy: 0.9934
 46/375 [==>...........................] - ETA: 2s - loss: 0.0235 - accuracy: 0.9932
 51/375 [===>..........................] - ETA: 2s - loss: 0.0227 - accuracy: 0.9934
 58/375 [===>..........................] - ETA: 2s - loss: 0.0225 - accuracy: 0.9934
 64/375 [====>.........................] - ETA: 2s - loss: 0.0233 - accuracy: 0.9930
 71/375 [====>.........................] - ETA: 2s - loss: 0.0229 - accuracy: 0.9934
 77/375 [=====>........................] - ETA: 2s - loss: 0.0226 - accuracy: 0.9933
 82/375 [=====>........................] - ETA: 2s - loss: 0.0229 - accuracy: 0.9934
 89/375 [======>.......................] - ETA: 2s - loss: 0.0229 - accuracy: 0.9935
 96/375 [======>.......................] - ETA: 2s - loss: 0.0230 - accuracy: 0.9934
102/375 [=======>......................] - ETA: 2s - loss: 0.0230 - accuracy: 0.9936
108/375 [=======>......................] - ETA: 2s - loss: 0.0230 - accuracy: 0.9933
114/375 [========>.....................] - ETA: 2s - loss: 0.0230 - accuracy: 0.9932
120/375 [========>.....................] - ETA: 2s - loss: 0.0224 - accuracy: 0.9934
126/375 [=========>....................] - ETA: 2s - loss: 0.0226 - accuracy: 0.9932
131/375 [=========>....................] - ETA: 2s - loss: 0.0228 - accuracy: 0.9931
137/375 [=========>....................] - ETA: 2s - loss: 0.0231 - accuracy: 0.9930
143/375 [==========>...................] - ETA: 2s - loss: 0.0229 - accuracy: 0.9931
149/375 [==========>...................] - ETA: 1s - loss: 0.0227 - accuracy: 0.9930
155/375 [===========>..................] - ETA: 1s - loss: 0.0225 - accuracy: 0.9930
161/375 [===========>..................] - ETA: 1s - loss: 0.0222 - accuracy: 0.9932
167/375 [============>.................] - ETA: 1s - loss: 0.0220 - accuracy: 0.9932
173/375 [============>.................] - ETA: 1s - loss: 0.0217 - accuracy: 0.9934
179/375 [=============>................] - ETA: 1s - loss: 0.0218 - accuracy: 0.9933
186/375 [=============>................] - ETA: 1s - loss: 0.0218 - accuracy: 0.9933
193/375 [==============>...............] - ETA: 1s - loss: 0.0217 - accuracy: 0.9934
199/375 [==============>...............] - ETA: 1s - loss: 0.0216 - accuracy: 0.9934
204/375 [===============>..............] - ETA: 1s - loss: 0.0219 - accuracy: 0.9933
210/375 [===============>..............] - ETA: 1s - loss: 0.0219 - accuracy: 0.9933
216/375 [================>.............] - ETA: 1s - loss: 0.0221 - accuracy: 0.9932
222/375 [================>.............] - ETA: 1s - loss: 0.0223 - accuracy: 0.9932
228/375 [=================>............] - ETA: 1s - loss: 0.0222 - accuracy: 0.9931
234/375 [=================>............] - ETA: 1s - loss: 0.0223 - accuracy: 0.9931
239/375 [==================>...........] - ETA: 1s - loss: 0.0224 - accuracy: 0.9931
246/375 [==================>...........] - ETA: 1s - loss: 0.0224 - accuracy: 0.9930
253/375 [===================>..........] - ETA: 1s - loss: 0.0227 - accuracy: 0.9928
259/375 [===================>..........] - ETA: 1s - loss: 0.0226 - accuracy: 0.9929
266/375 [====================>.........] - ETA: 0s - loss: 0.0225 - accuracy: 0.9930
273/375 [====================>.........] - ETA: 0s - loss: 0.0222 - accuracy: 0.9931
279/375 [=====================>........] - ETA: 0s - loss: 0.0224 - accuracy: 0.9929
285/375 [=====================>........] - ETA: 0s - loss: 0.0225 - accuracy: 0.9928
292/375 [======================>.......] - ETA: 0s - loss: 0.0226 - accuracy: 0.9928
299/375 [======================>.......] - ETA: 0s - loss: 0.0225 - accuracy: 0.9927
305/375 [=======================>......] - ETA: 0s - loss: 0.0226 - accuracy: 0.9927
311/375 [=======================>......] - ETA: 0s - loss: 0.0227 - accuracy: 0.9926
318/375 [========================>.....] - ETA: 0s - loss: 0.0224 - accuracy: 0.9927
324/375 [========================>.....] - ETA: 0s - loss: 0.0223 - accuracy: 0.9927
330/375 [=========================>....] - ETA: 0s - loss: 0.0225 - accuracy: 0.9928
336/375 [=========================>....] - ETA: 0s - loss: 0.0223 - accuracy: 0.9928
341/375 [==========================>...] - ETA: 0s - loss: 0.0222 - accuracy: 0.9929
346/375 [==========================>...] - ETA: 0s - loss: 0.0222 - accuracy: 0.9929
352/375 [===========================>..] - ETA: 0s - loss: 0.0223 - accuracy: 0.9929
357/375 [===========================>..] - ETA: 0s - loss: 0.0222 - accuracy: 0.9929
362/375 [===========================>..] - ETA: 0s - loss: 0.0220 - accuracy: 0.9930
367/375 [============================>.] - ETA: 0s - loss: 0.0220 - accuracy: 0.9930
372/375 [============================>.] - ETA: 0s - loss: 0.0220 - accuracy: 0.9930
375/375 [==============================] - 3s 9ms/step - loss: 0.0219 - accuracy: 0.9930

375/375 [==============================] - 5s 14ms/step - loss: 0.0219 - accuracy: 0.9930 - val_loss: 0.0636 - val_accuracy: 0.9824 - lr: 1.0000e-03
Epoch 13/20

  1/375 [..............................] - ETA: 6s - loss: 0.0297 - accuracy: 0.9922
  6/375 [..............................] - ETA: 4s - loss: 0.0170 - accuracy: 0.9948
 13/375 [>.............................] - ETA: 3s - loss: 0.0191 - accuracy: 0.9928
 19/375 [>.............................] - ETA: 3s - loss: 0.0170 - accuracy: 0.9947
 25/375 [=>............................] - ETA: 3s - loss: 0.0191 - accuracy: 0.9928
 31/375 [=>............................] - ETA: 3s - loss: 0.0180 - accuracy: 0.9932
 37/375 [=>............................] - ETA: 3s - loss: 0.0185 - accuracy: 0.9932
 43/375 [==>...........................] - ETA: 2s - loss: 0.0186 - accuracy: 0.9933
 49/375 [==>...........................] - ETA: 2s - loss: 0.0198 - accuracy: 0.9933
 54/375 [===>..........................] - ETA: 2s - loss: 0.0196 - accuracy: 0.9933
 60/375 [===>..........................] - ETA: 2s - loss: 0.0191 - accuracy: 0.9934
 66/375 [====>.........................] - ETA: 2s - loss: 0.0183 - accuracy: 0.9937
 70/375 [====>.........................] - ETA: 2s - loss: 0.0188 - accuracy: 0.9935
 76/375 [=====>........................] - ETA: 2s - loss: 0.0184 - accuracy: 0.9936
 82/375 [=====>........................] - ETA: 2s - loss: 0.0182 - accuracy: 0.9938
 88/375 [======>.......................] - ETA: 2s - loss: 0.0187 - accuracy: 0.9935
 94/375 [======>.......................] - ETA: 2s - loss: 0.0183 - accuracy: 0.9937
100/375 [=======>......................] - ETA: 2s - loss: 0.0177 - accuracy: 0.9940
106/375 [=======>......................] - ETA: 2s - loss: 0.0180 - accuracy: 0.9937
111/375 [=======>......................] - ETA: 2s - loss: 0.0181 - accuracy: 0.9936
117/375 [========>.....................] - ETA: 2s - loss: 0.0180 - accuracy: 0.9937
120/375 [========>.....................] - ETA: 2s - loss: 0.0180 - accuracy: 0.9938
126/375 [=========>....................] - ETA: 2s - loss: 0.0177 - accuracy: 0.9939
132/375 [=========>....................] - ETA: 2s - loss: 0.0183 - accuracy: 0.9937
138/375 [==========>...................] - ETA: 2s - loss: 0.0185 - accuracy: 0.9936
144/375 [==========>...................] - ETA: 2s - loss: 0.0184 - accuracy: 0.9936
150/375 [===========>..................] - ETA: 2s - loss: 0.0182 - accuracy: 0.9937
156/375 [===========>..................] - ETA: 2s - loss: 0.0179 - accuracy: 0.9938
162/375 [===========>..................] - ETA: 2s - loss: 0.0176 - accuracy: 0.9940
167/375 [============>.................] - ETA: 2s - loss: 0.0181 - accuracy: 0.9940
170/375 [============>.................] - ETA: 2s - loss: 0.0182 - accuracy: 0.9940
176/375 [=============>................] - ETA: 1s - loss: 0.0185 - accuracy: 0.9939
182/375 [=============>................] - ETA: 1s - loss: 0.0184 - accuracy: 0.9939
186/375 [=============>................] - ETA: 1s - loss: 0.0186 - accuracy: 0.9939
190/375 [==============>...............] - ETA: 1s - loss: 0.0185 - accuracy: 0.9940
195/375 [==============>...............] - ETA: 1s - loss: 0.0185 - accuracy: 0.9940
201/375 [===============>..............] - ETA: 1s - loss: 0.0186 - accuracy: 0.9939
207/375 [===============>..............] - ETA: 1s - loss: 0.0191 - accuracy: 0.9937
213/375 [================>.............] - ETA: 1s - loss: 0.0193 - accuracy: 0.9937
219/375 [================>.............] - ETA: 1s - loss: 0.0197 - accuracy: 0.9937
225/375 [=================>............] - ETA: 1s - loss: 0.0199 - accuracy: 0.9936
230/375 [=================>............] - ETA: 1s - loss: 0.0204 - accuracy: 0.9935
236/375 [=================>............] - ETA: 1s - loss: 0.0204 - accuracy: 0.9934
242/375 [==================>...........] - ETA: 1s - loss: 0.0205 - accuracy: 0.9934
247/375 [==================>...........] - ETA: 1s - loss: 0.0204 - accuracy: 0.9935
252/375 [===================>..........] - ETA: 1s - loss: 0.0204 - accuracy: 0.9935
258/375 [===================>..........] - ETA: 1s - loss: 0.0203 - accuracy: 0.9936
264/375 [====================>.........] - ETA: 1s - loss: 0.0202 - accuracy: 0.9936
270/375 [====================>.........] - ETA: 1s - loss: 0.0200 - accuracy: 0.9937
276/375 [=====================>........] - ETA: 0s - loss: 0.0201 - accuracy: 0.9937
280/375 [=====================>........] - ETA: 0s - loss: 0.0202 - accuracy: 0.9936
286/375 [=====================>........] - ETA: 0s - loss: 0.0203 - accuracy: 0.9935
293/375 [======================>.......] - ETA: 0s - loss: 0.0205 - accuracy: 0.9935
298/375 [======================>.......] - ETA: 0s - loss: 0.0205 - accuracy: 0.9934
304/375 [=======================>......] - ETA: 0s - loss: 0.0205 - accuracy: 0.9934
309/375 [=======================>......] - ETA: 0s - loss: 0.0204 - accuracy: 0.9935
315/375 [========================>.....] - ETA: 0s - loss: 0.0205 - accuracy: 0.9935
321/375 [========================>.....] - ETA: 0s - loss: 0.0204 - accuracy: 0.9935
326/375 [=========================>....] - ETA: 0s - loss: 0.0204 - accuracy: 0.9935
332/375 [=========================>....] - ETA: 0s - loss: 0.0205 - accuracy: 0.9935
336/375 [=========================>....] - ETA: 0s - loss: 0.0204 - accuracy: 0.9935
341/375 [==========================>...] - ETA: 0s - loss: 0.0204 - accuracy: 0.9936
346/375 [==========================>...] - ETA: 0s - loss: 0.0203 - accuracy: 0.9936
352/375 [===========================>..] - ETA: 0s - loss: 0.0203 - accuracy: 0.9936
358/375 [===========================>..] - ETA: 0s - loss: 0.0202 - accuracy: 0.9937
364/375 [============================>.] - ETA: 0s - loss: 0.0201 - accuracy: 0.9938
370/375 [============================>.] - ETA: 0s - loss: 0.0199 - accuracy: 0.9938
375/375 [==============================] - 4s 10ms/step - loss: 0.0201 - accuracy: 0.9938

375/375 [==============================] - 5s 12ms/step - loss: 0.0201 - accuracy: 0.9938 - val_loss: 0.0633 - val_accuracy: 0.9829 - lr: 1.0000e-03
Epoch 14/20

  1/375 [..............................] - ETA: 4s - loss: 0.0210 - accuracy: 0.9922
  6/375 [..............................] - ETA: 3s - loss: 0.0133 - accuracy: 0.9974
 12/375 [..............................] - ETA: 3s - loss: 0.0210 - accuracy: 0.9941
 18/375 [>.............................] - ETA: 3s - loss: 0.0195 - accuracy: 0.9939
 24/375 [>.............................] - ETA: 3s - loss: 0.0210 - accuracy: 0.9938
 30/375 [=>............................] - ETA: 3s - loss: 0.0213 - accuracy: 0.9945
 36/375 [=>............................] - ETA: 3s - loss: 0.0206 - accuracy: 0.9944
 42/375 [==>...........................] - ETA: 3s - loss: 0.0220 - accuracy: 0.9942
 47/375 [==>...........................] - ETA: 3s - loss: 0.0211 - accuracy: 0.9945
 54/375 [===>..........................] - ETA: 2s - loss: 0.0209 - accuracy: 0.9948
 59/375 [===>..........................] - ETA: 2s - loss: 0.0206 - accuracy: 0.9946
 65/375 [====>.........................] - ETA: 2s - loss: 0.0200 - accuracy: 0.9947
 71/375 [====>.........................] - ETA: 2s - loss: 0.0205 - accuracy: 0.9946
 77/375 [=====>........................] - ETA: 2s - loss: 0.0199 - accuracy: 0.9946
 82/375 [=====>........................] - ETA: 2s - loss: 0.0198 - accuracy: 0.9945
 88/375 [======>.......................] - ETA: 2s - loss: 0.0207 - accuracy: 0.9941
 94/375 [======>.......................] - ETA: 2s - loss: 0.0203 - accuracy: 0.9942
100/375 [=======>......................] - ETA: 2s - loss: 0.0198 - accuracy: 0.9945
106/375 [=======>......................] - ETA: 2s - loss: 0.0197 - accuracy: 0.9944
112/375 [=======>......................] - ETA: 2s - loss: 0.0194 - accuracy: 0.9944
114/375 [========>.....................] - ETA: 2s - loss: 0.0196 - accuracy: 0.9944
118/375 [========>.....................] - ETA: 2s - loss: 0.0195 - accuracy: 0.9943
123/375 [========>.....................] - ETA: 2s - loss: 0.0204 - accuracy: 0.9941
129/375 [=========>....................] - ETA: 2s - loss: 0.0201 - accuracy: 0.9942
135/375 [=========>....................] - ETA: 2s - loss: 0.0197 - accuracy: 0.9944
140/375 [==========>...................] - ETA: 2s - loss: 0.0195 - accuracy: 0.9944
146/375 [==========>...................] - ETA: 2s - loss: 0.0194 - accuracy: 0.9944
152/375 [===========>..................] - ETA: 2s - loss: 0.0192 - accuracy: 0.9944
158/375 [===========>..................] - ETA: 2s - loss: 0.0190 - accuracy: 0.9944
164/375 [============>.................] - ETA: 2s - loss: 0.0192 - accuracy: 0.9942
169/375 [============>.................] - ETA: 2s - loss: 0.0192 - accuracy: 0.9942
175/375 [=============>................] - ETA: 1s - loss: 0.0189 - accuracy: 0.9943
180/375 [=============>................] - ETA: 1s - loss: 0.0189 - accuracy: 0.9943
186/375 [=============>................] - ETA: 1s - loss: 0.0186 - accuracy: 0.9944
192/375 [==============>...............] - ETA: 1s - loss: 0.0186 - accuracy: 0.9944
197/375 [==============>...............] - ETA: 1s - loss: 0.0185 - accuracy: 0.9944
203/375 [===============>..............] - ETA: 1s - loss: 0.0186 - accuracy: 0.9944
206/375 [===============>..............] - ETA: 1s - loss: 0.0185 - accuracy: 0.9945
212/375 [===============>..............] - ETA: 1s - loss: 0.0185 - accuracy: 0.9944
219/375 [================>.............] - ETA: 1s - loss: 0.0184 - accuracy: 0.9944
225/375 [=================>............] - ETA: 1s - loss: 0.0184 - accuracy: 0.9943
229/375 [=================>............] - ETA: 1s - loss: 0.0185 - accuracy: 0.9943
234/375 [=================>............] - ETA: 1s - loss: 0.0186 - accuracy: 0.9943
240/375 [==================>...........] - ETA: 1s - loss: 0.0186 - accuracy: 0.9943
246/375 [==================>...........] - ETA: 1s - loss: 0.0187 - accuracy: 0.9942
252/375 [===================>..........] - ETA: 1s - loss: 0.0188 - accuracy: 0.9941
257/375 [===================>..........] - ETA: 1s - loss: 0.0188 - accuracy: 0.9941
262/375 [===================>..........] - ETA: 1s - loss: 0.0187 - accuracy: 0.9942
268/375 [====================>.........] - ETA: 1s - loss: 0.0186 - accuracy: 0.9942
274/375 [====================>.........] - ETA: 0s - loss: 0.0187 - accuracy: 0.9941
280/375 [=====================>........] - ETA: 0s - loss: 0.0187 - accuracy: 0.9941
286/375 [=====================>........] - ETA: 0s - loss: 0.0186 - accuracy: 0.9942
292/375 [======================>.......] - ETA: 0s - loss: 0.0185 - accuracy: 0.9942
298/375 [======================>.......] - ETA: 0s - loss: 0.0186 - accuracy: 0.9943
304/375 [=======================>......] - ETA: 0s - loss: 0.0186 - accuracy: 0.9943
309/375 [=======================>......] - ETA: 0s - loss: 0.0189 - accuracy: 0.9942
315/375 [========================>.....] - ETA: 0s - loss: 0.0191 - accuracy: 0.9941
321/375 [========================>.....] - ETA: 0s - loss: 0.0193 - accuracy: 0.9941
327/375 [=========================>....] - ETA: 0s - loss: 0.0194 - accuracy: 0.9941
333/375 [=========================>....] - ETA: 0s - loss: 0.0194 - accuracy: 0.9941
339/375 [==========================>...] - ETA: 0s - loss: 0.0194 - accuracy: 0.9941
345/375 [==========================>...] - ETA: 0s - loss: 0.0193 - accuracy: 0.9941
351/375 [===========================>..] - ETA: 0s - loss: 0.0193 - accuracy: 0.9941
356/375 [===========================>..] - ETA: 0s - loss: 0.0192 - accuracy: 0.9941
361/375 [===========================>..] - ETA: 0s - loss: 0.0193 - accuracy: 0.9940
366/375 [============================>.] - ETA: 0s - loss: 0.0194 - accuracy: 0.9940
369/375 [============================>.] - ETA: 0s - loss: 0.0193 - accuracy: 0.9940
374/375 [============================>.] - ETA: 0s - loss: 0.0195 - accuracy: 0.9940
375/375 [==============================] - 4s 10ms/step - loss: 0.0195 - accuracy: 0.9940

375/375 [==============================] - 5s 13ms/step - loss: 0.0195 - accuracy: 0.9940 - val_loss: 0.0632 - val_accuracy: 0.9827 - lr: 1.0000e-03
Epoch 15/20

  1/375 [..............................] - ETA: 3s - loss: 0.0065 - accuracy: 1.0000
  7/375 [..............................] - ETA: 3s - loss: 0.0117 - accuracy: 0.9978
 13/375 [>.............................] - ETA: 3s - loss: 0.0145 - accuracy: 0.9958
 20/375 [>.............................] - ETA: 2s - loss: 0.0159 - accuracy: 0.9941
 27/375 [=>............................] - ETA: 2s - loss: 0.0145 - accuracy: 0.9951
 33/375 [=>............................] - ETA: 2s - loss: 0.0135 - accuracy: 0.9955
 39/375 [==>...........................] - ETA: 2s - loss: 0.0157 - accuracy: 0.9948
 45/375 [==>...........................] - ETA: 2s - loss: 0.0177 - accuracy: 0.9944
 51/375 [===>..........................] - ETA: 2s - loss: 0.0188 - accuracy: 0.9942
 57/375 [===>..........................] - ETA: 2s - loss: 0.0192 - accuracy: 0.9938
 63/375 [====>.........................] - ETA: 2s - loss: 0.0185 - accuracy: 0.9940
 69/375 [====>.........................] - ETA: 2s - loss: 0.0182 - accuracy: 0.9941
 75/375 [=====>........................] - ETA: 2s - loss: 0.0186 - accuracy: 0.9942
 81/375 [=====>........................] - ETA: 2s - loss: 0.0183 - accuracy: 0.9943
 87/375 [=====>........................] - ETA: 2s - loss: 0.0187 - accuracy: 0.9942
 93/375 [======>.......................] - ETA: 2s - loss: 0.0189 - accuracy: 0.9941
 99/375 [======>.......................] - ETA: 2s - loss: 0.0184 - accuracy: 0.9943
103/375 [=======>......................] - ETA: 2s - loss: 0.0183 - accuracy: 0.9944
109/375 [=======>......................] - ETA: 2s - loss: 0.0189 - accuracy: 0.9943
115/375 [========>.....................] - ETA: 2s - loss: 0.0186 - accuracy: 0.9943
121/375 [========>.....................] - ETA: 2s - loss: 0.0186 - accuracy: 0.9943
127/375 [=========>....................] - ETA: 2s - loss: 0.0185 - accuracy: 0.9942
133/375 [=========>....................] - ETA: 2s - loss: 0.0181 - accuracy: 0.9944
139/375 [==========>...................] - ETA: 2s - loss: 0.0179 - accuracy: 0.9945
145/375 [==========>...................] - ETA: 2s - loss: 0.0179 - accuracy: 0.9946
151/375 [===========>..................] - ETA: 2s - loss: 0.0183 - accuracy: 0.9945
157/375 [===========>..................] - ETA: 1s - loss: 0.0186 - accuracy: 0.9943
163/375 [============>.................] - ETA: 1s - loss: 0.0186 - accuracy: 0.9941
169/375 [============>.................] - ETA: 1s - loss: 0.0187 - accuracy: 0.9941
175/375 [=============>................] - ETA: 1s - loss: 0.0187 - accuracy: 0.9942
181/375 [=============>................] - ETA: 1s - loss: 0.0191 - accuracy: 0.9940
187/375 [=============>................] - ETA: 1s - loss: 0.0191 - accuracy: 0.9940
193/375 [==============>...............] - ETA: 1s - loss: 0.0192 - accuracy: 0.9941
198/375 [==============>...............] - ETA: 1s - loss: 0.0191 - accuracy: 0.9941
204/375 [===============>..............] - ETA: 1s - loss: 0.0193 - accuracy: 0.9940
210/375 [===============>..............] - ETA: 1s - loss: 0.0190 - accuracy: 0.9941
216/375 [================>.............] - ETA: 1s - loss: 0.0191 - accuracy: 0.9941
222/375 [================>.............] - ETA: 1s - loss: 0.0192 - accuracy: 0.9940
228/375 [=================>............] - ETA: 1s - loss: 0.0195 - accuracy: 0.9939
233/375 [=================>............] - ETA: 1s - loss: 0.0193 - accuracy: 0.9940
239/375 [==================>...........] - ETA: 1s - loss: 0.0194 - accuracy: 0.9939
244/375 [==================>...........] - ETA: 1s - loss: 0.0195 - accuracy: 0.9938
249/375 [==================>...........] - ETA: 1s - loss: 0.0196 - accuracy: 0.9938
255/375 [===================>..........] - ETA: 1s - loss: 0.0196 - accuracy: 0.9938
261/375 [===================>..........] - ETA: 1s - loss: 0.0198 - accuracy: 0.9937
267/375 [====================>.........] - ETA: 0s - loss: 0.0199 - accuracy: 0.9936
273/375 [====================>.........] - ETA: 0s - loss: 0.0197 - accuracy: 0.9937
279/375 [=====================>........] - ETA: 0s - loss: 0.0199 - accuracy: 0.9936
285/375 [=====================>........] - ETA: 0s - loss: 0.0200 - accuracy: 0.9936
291/375 [======================>.......] - ETA: 0s - loss: 0.0199 - accuracy: 0.9936
297/375 [======================>.......] - ETA: 0s - loss: 0.0200 - accuracy: 0.9935
302/375 [=======================>......] - ETA: 0s - loss: 0.0201 - accuracy: 0.9935
308/375 [=======================>......] - ETA: 0s - loss: 0.0199 - accuracy: 0.9936
314/375 [========================>.....] - ETA: 0s - loss: 0.0200 - accuracy: 0.9935
317/375 [========================>.....] - ETA: 0s - loss: 0.0199 - accuracy: 0.9936
322/375 [========================>.....] - ETA: 0s - loss: 0.0198 - accuracy: 0.9936
326/375 [=========================>....] - ETA: 0s - loss: 0.0196 - accuracy: 0.9937
332/375 [=========================>....] - ETA: 0s - loss: 0.0196 - accuracy: 0.9937
338/375 [==========================>...] - ETA: 0s - loss: 0.0195 - accuracy: 0.9938
344/375 [==========================>...] - ETA: 0s - loss: 0.0194 - accuracy: 0.9938
350/375 [===========================>..] - ETA: 0s - loss: 0.0195 - accuracy: 0.9938
356/375 [===========================>..] - ETA: 0s - loss: 0.0195 - accuracy: 0.9938
362/375 [===========================>..] - ETA: 0s - loss: 0.0194 - accuracy: 0.9938
368/375 [============================>.] - ETA: 0s - loss: 0.0194 - accuracy: 0.9937
374/375 [============================>.] - ETA: 0s - loss: 0.0194 - accuracy: 0.9938
375/375 [==============================] - 4s 9ms/step - loss: 0.0194 - accuracy: 0.9937

375/375 [==============================] - 4s 12ms/step - loss: 0.0194 - accuracy: 0.9937 - val_loss: 0.0631 - val_accuracy: 0.9827 - lr: 1.0000e-03
Epoch 16/20

  1/375 [..............................] - ETA: 3s - loss: 0.0555 - accuracy: 0.9766
  7/375 [..............................] - ETA: 3s - loss: 0.0296 - accuracy: 0.9877
 13/375 [>.............................] - ETA: 3s - loss: 0.0225 - accuracy: 0.9916
 19/375 [>.............................] - ETA: 3s - loss: 0.0259 - accuracy: 0.9901
 25/375 [=>............................] - ETA: 3s - loss: 0.0242 - accuracy: 0.9909
 31/375 [=>............................] - ETA: 3s - loss: 0.0249 - accuracy: 0.9912
 36/375 [=>............................] - ETA: 3s - loss: 0.0238 - accuracy: 0.9915
 42/375 [==>...........................] - ETA: 3s - loss: 0.0247 - accuracy: 0.9916
 48/375 [==>...........................] - ETA: 2s - loss: 0.0242 - accuracy: 0.9917
 51/375 [===>..........................] - ETA: 3s - loss: 0.0232 - accuracy: 0.9922
 57/375 [===>..........................] - ETA: 3s - loss: 0.0233 - accuracy: 0.9921
 63/375 [====>.........................] - ETA: 2s - loss: 0.0231 - accuracy: 0.9918
 69/375 [====>.........................] - ETA: 2s - loss: 0.0228 - accuracy: 0.9922
 74/375 [====>.........................] - ETA: 2s - loss: 0.0227 - accuracy: 0.9923
 80/375 [=====>........................] - ETA: 2s - loss: 0.0225 - accuracy: 0.9925
 86/375 [=====>........................] - ETA: 2s - loss: 0.0221 - accuracy: 0.9928
 91/375 [======>.......................] - ETA: 2s - loss: 0.0225 - accuracy: 0.9928
 97/375 [======>.......................] - ETA: 2s - loss: 0.0232 - accuracy: 0.9925
103/375 [=======>......................] - ETA: 2s - loss: 0.0225 - accuracy: 0.9928
108/375 [=======>......................] - ETA: 2s - loss: 0.0225 - accuracy: 0.9928
114/375 [========>.....................] - ETA: 2s - loss: 0.0232 - accuracy: 0.9925
120/375 [========>.....................] - ETA: 2s - loss: 0.0227 - accuracy: 0.9928
125/375 [=========>....................] - ETA: 2s - loss: 0.0224 - accuracy: 0.9929
131/375 [=========>....................] - ETA: 2s - loss: 0.0222 - accuracy: 0.9928
136/375 [=========>....................] - ETA: 2s - loss: 0.0220 - accuracy: 0.9928
142/375 [==========>...................] - ETA: 2s - loss: 0.0220 - accuracy: 0.9928
148/375 [==========>...................] - ETA: 2s - loss: 0.0218 - accuracy: 0.9928
154/375 [===========>..................] - ETA: 2s - loss: 0.0219 - accuracy: 0.9928
160/375 [===========>..................] - ETA: 2s - loss: 0.0214 - accuracy: 0.9930
166/375 [============>.................] - ETA: 2s - loss: 0.0211 - accuracy: 0.9932
172/375 [============>.................] - ETA: 1s - loss: 0.0211 - accuracy: 0.9932
178/375 [=============>................] - ETA: 1s - loss: 0.0208 - accuracy: 0.9932
184/375 [=============>................] - ETA: 1s - loss: 0.0209 - accuracy: 0.9932
190/375 [==============>...............] - ETA: 1s - loss: 0.0210 - accuracy: 0.9932
197/375 [==============>...............] - ETA: 1s - loss: 0.0209 - accuracy: 0.9933
203/375 [===============>..............] - ETA: 1s - loss: 0.0212 - accuracy: 0.9931
208/375 [===============>..............] - ETA: 1s - loss: 0.0210 - accuracy: 0.9932
214/375 [================>.............] - ETA: 1s - loss: 0.0209 - accuracy: 0.9932
220/375 [================>.............] - ETA: 1s - loss: 0.0206 - accuracy: 0.9933
226/375 [=================>............] - ETA: 1s - loss: 0.0210 - accuracy: 0.9932
232/375 [=================>............] - ETA: 1s - loss: 0.0209 - accuracy: 0.9932
238/375 [==================>...........] - ETA: 1s - loss: 0.0210 - accuracy: 0.9932
242/375 [==================>...........] - ETA: 1s - loss: 0.0208 - accuracy: 0.9933
246/375 [==================>...........] - ETA: 1s - loss: 0.0208 - accuracy: 0.9933
251/375 [===================>..........] - ETA: 1s - loss: 0.0206 - accuracy: 0.9934
257/375 [===================>..........] - ETA: 1s - loss: 0.0206 - accuracy: 0.9934
261/375 [===================>..........] - ETA: 1s - loss: 0.0206 - accuracy: 0.9934
267/375 [====================>.........] - ETA: 1s - loss: 0.0204 - accuracy: 0.9935
272/375 [====================>.........] - ETA: 1s - loss: 0.0206 - accuracy: 0.9933
278/375 [=====================>........] - ETA: 0s - loss: 0.0205 - accuracy: 0.9934
284/375 [=====================>........] - ETA: 0s - loss: 0.0205 - accuracy: 0.9933
290/375 [======================>.......] - ETA: 0s - loss: 0.0206 - accuracy: 0.9934
296/375 [======================>.......] - ETA: 0s - loss: 0.0207 - accuracy: 0.9934
303/375 [=======================>......] - ETA: 0s - loss: 0.0206 - accuracy: 0.9935
308/375 [=======================>......] - ETA: 0s - loss: 0.0205 - accuracy: 0.9934
314/375 [========================>.....] - ETA: 0s - loss: 0.0206 - accuracy: 0.9935
320/375 [========================>.....] - ETA: 0s - loss: 0.0205 - accuracy: 0.9935
325/375 [=========================>....] - ETA: 0s - loss: 0.0205 - accuracy: 0.9934
331/375 [=========================>....] - ETA: 0s - loss: 0.0207 - accuracy: 0.9934
337/375 [=========================>....] - ETA: 0s - loss: 0.0206 - accuracy: 0.9935
343/375 [==========================>...] - ETA: 0s - loss: 0.0204 - accuracy: 0.9935
349/375 [==========================>...] - ETA: 0s - loss: 0.0204 - accuracy: 0.9935
355/375 [===========================>..] - ETA: 0s - loss: 0.0202 - accuracy: 0.9936
361/375 [===========================>..] - ETA: 0s - loss: 0.0201 - accuracy: 0.9937
367/375 [============================>.] - ETA: 0s - loss: 0.0200 - accuracy: 0.9937
373/375 [============================>.] - ETA: 0s - loss: 0.0200 - accuracy: 0.9937
375/375 [==============================] - 4s 10ms/step - loss: 0.0200 - accuracy: 0.9937

375/375 [==============================] - 5s 12ms/step - loss: 0.0200 - accuracy: 0.9937 - val_loss: 0.0631 - val_accuracy: 0.9826 - lr: 1.0000e-04
Epoch 17/20

  1/375 [..............................] - ETA: 3s - loss: 0.0035 - accuracy: 1.0000
  7/375 [..............................] - ETA: 3s - loss: 0.0316 - accuracy: 0.9888
 13/375 [>.............................] - ETA: 3s - loss: 0.0236 - accuracy: 0.9916
 19/375 [>.............................] - ETA: 3s - loss: 0.0214 - accuracy: 0.9918
 25/375 [=>............................] - ETA: 3s - loss: 0.0209 - accuracy: 0.9925
 31/375 [=>............................] - ETA: 3s - loss: 0.0206 - accuracy: 0.9922
 36/375 [=>............................] - ETA: 3s - loss: 0.0201 - accuracy: 0.9924
 42/375 [==>...........................] - ETA: 3s - loss: 0.0224 - accuracy: 0.9916
 48/375 [==>...........................] - ETA: 3s - loss: 0.0217 - accuracy: 0.9922
 53/375 [===>..........................] - ETA: 3s - loss: 0.0216 - accuracy: 0.9925
 59/375 [===>..........................] - ETA: 3s - loss: 0.0208 - accuracy: 0.9930
 65/375 [====>.........................] - ETA: 2s - loss: 0.0209 - accuracy: 0.9933
 71/375 [====>.........................] - ETA: 2s - loss: 0.0212 - accuracy: 0.9934
 77/375 [=====>........................] - ETA: 2s - loss: 0.0203 - accuracy: 0.9937
 80/375 [=====>........................] - ETA: 2s - loss: 0.0203 - accuracy: 0.9938
 85/375 [=====>........................] - ETA: 2s - loss: 0.0203 - accuracy: 0.9937
 91/375 [======>.......................] - ETA: 2s - loss: 0.0203 - accuracy: 0.9936
 97/375 [======>.......................] - ETA: 2s - loss: 0.0200 - accuracy: 0.9937
102/375 [=======>......................] - ETA: 2s - loss: 0.0199 - accuracy: 0.9938
108/375 [=======>......................] - ETA: 2s - loss: 0.0198 - accuracy: 0.9936
114/375 [========>.....................] - ETA: 2s - loss: 0.0203 - accuracy: 0.9935
120/375 [========>.....................] - ETA: 2s - loss: 0.0199 - accuracy: 0.9936
126/375 [=========>....................] - ETA: 2s - loss: 0.0200 - accuracy: 0.9938
132/375 [=========>....................] - ETA: 2s - loss: 0.0202 - accuracy: 0.9937
138/375 [==========>...................] - ETA: 2s - loss: 0.0201 - accuracy: 0.9936
144/375 [==========>...................] - ETA: 2s - loss: 0.0200 - accuracy: 0.9936
150/375 [===========>..................] - ETA: 2s - loss: 0.0200 - accuracy: 0.9936
156/375 [===========>..................] - ETA: 2s - loss: 0.0203 - accuracy: 0.9934
162/375 [===========>..................] - ETA: 2s - loss: 0.0201 - accuracy: 0.9935
167/375 [============>.................] - ETA: 2s - loss: 0.0197 - accuracy: 0.9936
172/375 [============>.................] - ETA: 1s - loss: 0.0202 - accuracy: 0.9934
178/375 [=============>................] - ETA: 1s - loss: 0.0203 - accuracy: 0.9934
183/375 [=============>................] - ETA: 1s - loss: 0.0202 - accuracy: 0.9935
189/375 [==============>...............] - ETA: 1s - loss: 0.0204 - accuracy: 0.9934
195/375 [==============>...............] - ETA: 1s - loss: 0.0203 - accuracy: 0.9934
201/375 [===============>..............] - ETA: 1s - loss: 0.0207 - accuracy: 0.9932
207/375 [===============>..............] - ETA: 1s - loss: 0.0207 - accuracy: 0.9932
212/375 [===============>..............] - ETA: 1s - loss: 0.0212 - accuracy: 0.9931
218/375 [================>.............] - ETA: 1s - loss: 0.0212 - accuracy: 0.9931
223/375 [================>.............] - ETA: 1s - loss: 0.0211 - accuracy: 0.9932
229/375 [=================>............] - ETA: 1s - loss: 0.0208 - accuracy: 0.9933
234/375 [=================>............] - ETA: 1s - loss: 0.0207 - accuracy: 0.9934
240/375 [==================>...........] - ETA: 1s - loss: 0.0206 - accuracy: 0.9934
246/375 [==================>...........] - ETA: 1s - loss: 0.0205 - accuracy: 0.9935
251/375 [===================>..........] - ETA: 1s - loss: 0.0205 - accuracy: 0.9934
256/375 [===================>..........] - ETA: 1s - loss: 0.0204 - accuracy: 0.9935
261/375 [===================>..........] - ETA: 1s - loss: 0.0205 - accuracy: 0.9934
267/375 [====================>.........] - ETA: 1s - loss: 0.0206 - accuracy: 0.9934
273/375 [====================>.........] - ETA: 0s - loss: 0.0206 - accuracy: 0.9934
278/375 [=====================>........] - ETA: 0s - loss: 0.0205 - accuracy: 0.9935
284/375 [=====================>........] - ETA: 0s - loss: 0.0206 - accuracy: 0.9934
290/375 [======================>.......] - ETA: 0s - loss: 0.0205 - accuracy: 0.9934
296/375 [======================>.......] - ETA: 0s - loss: 0.0204 - accuracy: 0.9935
302/375 [=======================>......] - ETA: 0s - loss: 0.0203 - accuracy: 0.9935
307/375 [=======================>......] - ETA: 0s - loss: 0.0201 - accuracy: 0.9936
312/375 [=======================>......] - ETA: 0s - loss: 0.0202 - accuracy: 0.9935
318/375 [========================>.....] - ETA: 0s - loss: 0.0202 - accuracy: 0.9935
323/375 [========================>.....] - ETA: 0s - loss: 0.0202 - accuracy: 0.9935
328/375 [=========================>....] - ETA: 0s - loss: 0.0202 - accuracy: 0.9935
334/375 [=========================>....] - ETA: 0s - loss: 0.0200 - accuracy: 0.9936
340/375 [==========================>...] - ETA: 0s - loss: 0.0199 - accuracy: 0.9937
346/375 [==========================>...] - ETA: 0s - loss: 0.0201 - accuracy: 0.9937
352/375 [===========================>..] - ETA: 0s - loss: 0.0201 - accuracy: 0.9936
357/375 [===========================>..] - ETA: 0s - loss: 0.0202 - accuracy: 0.9936
362/375 [===========================>..] - ETA: 0s - loss: 0.0201 - accuracy: 0.9936
368/375 [============================>.] - ETA: 0s - loss: 0.0202 - accuracy: 0.9935
373/375 [============================>.] - ETA: 0s - loss: 0.0202 - accuracy: 0.9935
375/375 [==============================] - 4s 10ms/step - loss: 0.0202 - accuracy: 0.9935

375/375 [==============================] - 5s 12ms/step - loss: 0.0202 - accuracy: 0.9935 - val_loss: 0.0631 - val_accuracy: 0.9826 - lr: 1.0000e-05

history

plot(history)

Repeat

At this point, we have a pretty good model. However, often, iterating over these steps can improve model performance even further. For brevity, we’ll act as if we have found a sufficient solution.

Evaluate results

Once a final model is chosen, we can evaluate the model on our test set to provide us with a accurate expectation of our generalization error. Our goal is that our test error is very close to our validation error.

# model %>% evaluate(test_images, test_labels, verbose = FALSE)

Confusion matrix

To understand our model’s performance across the different response classes, we can assess a confusion matrix ℹ️.

First, we need to predict our classes and also get the actual response values.

# predictions <- model %>% predict(test_images)
actual <- mnist$test$y

We can see the number of missed predictions in our test set

missed_predictions <- sum(predictions != actual)
missed_predictions

We can use caret::confusionMatrix() to get our confusion matrix. We can see which digits our model confuses the most by analyzing the confusion matrix.

• 9s are often confused with 4s
• 6s are often confused with 0s & 5s
• etc.

caret::confusionMatrix(factor(predictions), factor(actual))

Visualize missed predictions

We can also visualize this with the following:

tibble(
  actual,
  predictions
  ) %>% 
  filter(actual != predictions) %>%
  count(actual, predictions) %>%
  mutate(perc = n / n() * 100) %>% 
  filter(n > 1) %>% 
  ggplot(aes(actual, predictions, size = n)) +
  geom_point(shape = 15, col = "#9F92C6") +
  scale_x_continuous("Actual Target", breaks = 0:9) +
  scale_y_continuous("Prediction", breaks = 0:9) +
  scale_size_area(breaks = c(2, 5, 10, 15), max_size = 5) +
  coord_fixed() +
  ggtitle(paste(missed_predictions, "mismatches")) +
  theme(panel.grid.minor = element_blank()) +
  labs(caption = 'Adapted from Rick Scavetta')

Visualize missed predictions

Lastly, lets check out those mispredicted digits.

missed <- which(predictions != actual)
plot_dim <- ceiling(sqrt(length(missed)))

par(mfrow = c(plot_dim, plot_dim), mar = c(0,0,0,0))
for (i in missed) {
  plot(as.raster(mnist$test$x[i,,], max = 255))
}

If we look at the predicted vs actual we can reason about why our model mispredicted some of the digits.

par(mfrow = c(4, 4), mar = c(0,0,2,0))

for (i in missed[1:16]) {
  plot(as.raster(mnist$test$x[i,,], max = 255)) 
  title(main = paste("Predicted:", predictions[i]))
}

Key takeaways

Follow these steps and guidelines when tuning your DL model:

1. Prepare data
   • data needs to be shaped into the right tensor dimensions
   • data should be scaled so they don’t trigger exploding gradients
2. Balance batch size with a default learning rate so that…
   • your learning curve is not too noisy
   • the training compute time is sufficient
3. Tune the adaptive learning rate optimizer
   • compare SGD+momentum with RMSprop & Adam
   • assess learning rates on log scale between [1e-1, 1e-7]
4. Add callbacks to control training
   • use early stopping for more efficient training
   • use an adaptive learning rate callback to have more control of the learning rate
5. Explore model capacity
   • compare adding width vs depth (consider loss vs. training time)
   • be smart with your experiments (use tensorboard callback!)
6. Regularize overfitting
   • weight decay controls magnitude of weights; start by assessing values btwn 0.1, 0.01, …, 0.0001
   • dropout minimizes happenstance patterns from noise; typical values range from 0.2-0.5.
7. Iterate, iterate, iterate!

🏠